Senior Data Engineer

Responsibilities:

• Own the reliability and performance of operational pipelines across our product catalog infrastructure.
• Identify and reduce technical debt, replacing reactive patches with designed, testable logic.
• Build and maintain low-latency data APIs that serve downstream operational and analytics consumers.
• Implement CDC patterns to keep catalog data synchronized across systems with minimal lag.
• Build monitoring, observability, and automated testing so failures surface before stakeholders report them.
• Design and implement unit standardization and master data logic at the catalog scale.
• Translate business requirements from non-technical stakeholders into durable pipeline logic.
• Own code versioning, deployment, and incident response for your layer.
• Leverage your expertise in the tech stack: BigQuery, Databricks, Spark/PySpark, and AWS/GCP.

Requirements:

• 3+ years as a data engineer, including solo or primary ownership of production pipelines.
• Strong Python; data engineering, transformation logic, and testing discipline.
• Strong SQL with the ability to write correct queries and identify and refactor anti-patterns.
• Databricks, Delta Lake, and Airflow for production orchestration.
• Experience with CDC patterns for real-time or near-real-time data synchronization.
• Experience building low-latency APIs serving operational or analytical consumers.
• Test-driven development discipline: unit tests, integration tests, and regression coverage as standard practice, not an afterthought.
• Operates independently under ambiguity; designs systems to be maintained, not just to run.

Preferred:

• Kafka or equivalent event streaming platform experience.
• Experience with entity matching, deduplication, or master data management.
• Exposure to ML pipeline support in production.
• Familiarity with NLP techniques for entity resolution or text normalization (tokenization, similarity matching, named entity recognition).