Senior Data Engineer

Role : Strong Azure Databricks Engineer / Senior Data Engineer Profile

1. Mandatory (Experience 1) :

- Must have minimum 8+ years of overall experience in Data Engineering, Data Development, or related data technology roles, with strong hands-on experience in enterprise data pipeline development.

2. Mandatory (Experience 2) :

- Must have strong hands-on experience with Azure Databricks, including development and optimization of scalable data engineering workflows using Databricks and PySpark.

3. Mandatory (Experience 3) :

- Must have strong hands-on proficiency in PySpark/Python and SQL, with proven experience developing complex data transformations, processing workflows, and performance-optimized queries.

4. Mandatory (Experience 4) :

- Must have hands-on experience with Azure Data Factory (ADF) for designing, developing, and orchestrating end-to-end data pipelines and integrating data from multiple sources.

5. Mandatory (Experience 5) :

- Must have strong experience working on the Azure Cloud platform and associated data services, with solid understanding of data warehousing, data modeling, pipeline architecture, and enterprise data solutions.

6. Mandatory (Experience 6) :

- Must have hands-on experience implementing CI/CD using Azure DevOps, including deployment and release management across development, QA, and production environments.

7. Mandatory (Experience 7) :

- Must have proven technical leadership experience, including code reviews, enforcing development best practices, mentoring developers, and providing technical guidance to a data engineering team.

8. Mandatory (Notice Period) :

- Immediate joiners or candidates who can join within 15 days.

9. Mandatory (Note) :

- The position is open across all Cognizant offices pan India. Candidates must be willing to attend the F2F interview at the nearest Cognizant office location.

Key Responsibilities :

- Architect and implement scalable ETL/ELT pipelines using PySpark and Azure Databricks to process large-scale datasets efficiently.

- Optimize existing data workflows and Spark jobs to improve performance, reduce latency, and minimize operational costs for client projects.

- Design robust data models and storage architectures within the Azure Data Lake environment to support advanced analytics and reporting needs.

- Collaborate with data architects and business analysts to define technical specifications and ensure alignment with organizational data governance standards.

- Mentor junior team members and conduct code reviews to maintain high standards of software engineering and data integrity across the development lifecycle.

Skills

Data Engineering, Azure Databricks, PySpark, Python, SQL, Azure Data Factory, ETL, Data Pipeline