Senior Data Engineer

• Design, build, maintain, and document scalable data pipelines and ingestion frameworks.
• Develop and optimize batch and real-time data processing solutions.
• Implement and support modern data lakehouse architectures using Delta Lake and cloud-native technologies.
• Build and manage data streaming solutions using Kafka, Spark Streaming, and Change Data Capture (CDC) workflows.
• Ensure high standards of data quality, governance, security, and reliability across data platforms.
• Collaborate closely with Data Scientists, Machine Learning Engineers, BI teams, and Platform Engineering teams to deliver data solutions.
• Optimize data workflows for performance, scalability, and cost efficiency.
• Lead technical design discussions and contribute to architectural decisions.
• Mentor junior and mid-level engineers, providing technical guidance and fostering team growth.
• Promote data engineering best practices, coding standards, and operational excellence.

Requirements

• 5-8+ years of experience in Data Engineering or a related field.
• Strong expertise in cloud-based data platforms, particularly Databricks.
• Deep understanding of Data Lake and Lakehouse architectures, including Delta Lake.
• Hands-on experience with real-time data streaming technologies such as Kafka, Apache Spark, and CDC frameworks.
• Strong programming skills in Python, PySpark, and PL/SQL.Experience with containerization and orchestration technologies including Docker and Kubernetes.
• Hands-on experience developing and managing Apache Airflow DAGs.
• Proficiency with Git and modern software development practices.
• Strong analytical, problem-solving, and troubleshooting skills.
• Excellent communication and stakeholder management abilities.