Sr Data Engineer

About Us:

Narwal, with its Global Delivery Model, strategically expands its reach across North America, the United Kingdom, and an offshore development centre in India. Delivery cutting edge AI, Data and Quality Engineering solutions and consistently surpassing expectations, Narwal has achieved remarkable triple-digit growth rates year after year, earning accolades such as Inc. 5000, Best IT Services Company, Best Data Technology Company, and Partner of the Year with Tricentis.

Our Vision:

To be an expert in AI, Data and Quality Engineering transformations, bold in our thinking and authentic in our relationships.

Title: Sr Data Engineer

Experience Level : 6-12 years

Must Have Skills - Snowflake, DBT, AWS OR GCP, Airflow, Kafka, Fivetran

Work Location- Pune ( Work from Office)

About the Role

We are looking for an experienced Data Engineer with 6+ years of hands-on experience in designing, developing, and operationalizing enterprise data ingestion pipelines and trusted data products within a Data Lake ecosystem. The ideal candidate will have strong experience across data ingestion, transformation, orchestration, data governance, data quality, and production operations. The role requires hands-on expertise with technologies such as Fivetran, Kafka, Apache Iceberg, Snowflake, dbt, and Astronomer/Airflow, along with exposure to data observability using Monte Carlo.

Key Responsibilities

• Lead the design, development, and implementation of enterprise data ingestion initiatives using Fivetran, Kafka, and Apache Iceberg.
• Design and build scalable, reliable batch and near-real-time ingestion pipelines to integrate data from diverse source systems into the Data Lake ecosystem.
• Develop and maintain ingestion frameworks and data pipelines, ensuring performance, scalability, and operational reliability.
• Build and optimize data transformation pipelines using dbt, following modularity, reusability, testing, and documentation best practices.
• Build, schedule, monitor, and troubleshoot workflows using Apache Airflow and Astronomer.
• Develop reusable and modular Airflow DAGs and support the management and operation of Airflow environments.
• Develop and maintain Trusted Data Layer assets, ensuring data is validated, consistent, and ready for downstream consumption.
• Publish curated datasets to Snowflake and support efficient data models for analytics and other downstream use cases.
• Partner with stakeholders to implement data cataloging, classification, metadata management, lineage, and other data governance requirements.
• Implement data quality tests, validation rules, and reconciliation mechanisms to ensure data accuracy, consistency, and reliability.
• Configure, implement, and maintain Monte Carlo for data observability, monitoring, anomaly detection, and alerting.
• Monitor pipeline health, proactively identify data quality or pipeline issues, and drive timely resolution.
• Collaborate with SRE teams to support production operations, incident management, root cause analysis, and platform reliability.
• Implement appropriate logging, alerting, error handling, and recovery mechanisms for production-grade data pipelines.
• Document technical solutions, data workflows, operational procedures, and provide technical leadership throughout the delivery lifecycle.

Must-Have Skills

• 6+ years of overall experience in Data Engineering, with hands-on experience building and supporting production-grade data pipelines.
• Strong hands-on experience with Fivetran for data ingestion and integration from multiple source systems.
• Experience with Apache Kafka for event-driven and near-real-time data ingestion.
• Hands-on experience with Apache Iceberg and a good understanding of open table formats and Data Lake/Lakehouse concepts.
• Strong hands-on experience with Snowflake, including data modeling and publishing curated datasets.
• Strong hands-on experience with dbt for data transformation, models, tests, and documentation.
• Strong experience with Apache Airflow for workflow orchestration and DAG development.
• Experience working with Astronomer and managing or operating Airflow environments.
• Experience with Monte Carlo for data observability, monitoring, anomaly detection, and alerting.
• Good understanding of data governance concepts, including data cataloging, classification, metadata management, and lineage.
• Experience implementing data quality checks, validation, and testing frameworks.
• Experience supporting production data platforms, troubleshooting pipeline failures, and participating in incident resolution.
• Strong SQL skills and understanding of ETL/ELT, data integration, and data modeling concepts.
• Experience with Git, version control, code reviews, and standard software engineering practices.
• Exposure to cloud platforms such as Google Cloud Platform (GCP) or Amazon Web Services (AWS) is preferred.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• Equivalent technical experience may also be considered.