Lead/Sr Data Engineer

About Us:

Narwal, with its Global Delivery Model, strategically expands its reach across North America, the United Kingdom, and an offshore development centre in India. Delivery cutting edge AI, Data and Quality Engineering solutions and consistently surpassing expectations, Narwal has achieved remarkable triple-digit growth rates year after year, earning accolades such as Inc. 5000, Best IT Services Company, Best Data Technology Company, and Partner of the Year with Tricentis.

Our Vision:

To be an expert in AI, Data and Quality Engineering transformations, bold in our thinking and authentic in our relationships.

Role- Lead/Sr Data Engineer

Experience level - 8-12Yrs

Work Location- Pune ( Work from Office)

Must Have skills- Snowflake, DBT, Airflow, AWS/GCP

Looking for Immediate joiners

About the Role

We are looking for an experienced Data Engineer with 6 - 9 years of hands-on experience in designing, developing, and maintaining scalable data pipelines and data platforms.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for batch and near-real-time data processing.
• Develop and optimize data models and transformation pipelines using Snowflake and dbt (Data build Tool).
• Build, schedule, monitor, and troubleshoot workflows using Apache Airflow.
• Work with Astronomer for managing and operating Airflow environments.
• Develop reusable and modular DAGs following Airflow best practices.
• Design efficient data warehouse solutions using Snowflake, including databases, schemas, tables, views, stages, and related objects.
• Develop complex SQL queries and optimize them for performance and cost.
• Implement data transformation, cleansing, validation, and enrichment processes using dbt (Data build Tool).
• Develop and maintain dbt models, tests, macros, snapshots, and documentation.
• Work with cloud services on GCP or AWS to build and support data engineering solutions.
• Integrate data from multiple sources into Snowflake and downstream data platforms.
• Implement data quality checks and ensure accuracy, consistency, and reliability of data.
• Monitor data pipelines and proactively troubleshoot failures and performance issues.
• Optimize Snowflake workloads, queries, storage, and compute usage.
• Implement appropriate logging, monitoring, alerting, and error-handling mechanisms.
• Work with CI/CD processes for deploying data engineering code across environments.
• Follow best practices for version control, code reviews, testing, and deployment.
• Collaborate with architects, analysts, data scientists, application teams, and business stakeholders to understand data requirements.
• Participate in technical discussions and contribute to data platform architecture and design decisions.
• Mentor junior and mid-level data engineers and provide technical guidance where required.

Must-Have Skills

• 6 – 9 years of overall experience in Data Engineering.
• Strong hands-on experience with Snowflake.
• Strong hands-on experience with dbt (Data Build Tool).
• Strong experience with Apache Airflow for workflow orchestration.
• Experience working with Astronomer and managing Airflow environments.
• Strong SQL skills, including complex queries, joins, CTEs, window functions, and query optimization.
• Hands-on experience with at least one cloud platform:
• Google Cloud Platform (GCP), or
• Amazon Web Services (AWS)
• Experience designing and supporting production-grade data pipelines.
• Good understanding of data warehousing and data modeling concepts.
• Experience with Git and standard software development practices.
• Good understanding of ETL/ELT concepts and data integration.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

Equivalent technical experience may also be considered.