Azure Data Engineer

Job Title : Azure Data Engineer (7+ Years Experience)

Location : Hyderabad - Hybrid

Experience : 7+ Years

Employment Type : Full-Time

Job Summary:

We are looking for an experienced Azure Data Engineer with 7+ years of expertise in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong experience with Azure data services, ETL/ELT pipelines, cloud data warehousing, big data processing, and SQL/Python development. The role involves collaborating with cross-functional teams to build reliable, secure, and high-performance data platforms.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using Azure data services.

- Build and optimize ETL/ELT processes using Azure Data Factory (ADF).

- Develop data integration solutions using Azure Databricks, PySpark, and Spark SQL.

- Design and implement data lakes using Azure Data Lake Storage Gen2 (ADLS).

- Develop and maintain cloud-based data warehouse solutions using Azure Synapse Analytics.

- Create and optimize SQL queries, stored procedures, and database objects.

- Work with structured, semi-structured, and unstructured data from multiple sources.

- Implement data quality, validation, governance, and security best practices.

- Optimize data processing performance and troubleshoot pipeline failures.

- Automate deployments using Azure DevOps and CI/CD pipelines.

- Monitor Azure resources and ensure high availability and performance.

- Collaborate with business analysts, data scientists, and application teams to deliver data solutions.

- Prepare technical documentation and provide production support.

Required Skills :

1. Azure Services :

- Azure Data Factory (ADF)

- Azure Databricks

- Azure Synapse Analytics

- Azure Data Lake Storage Gen2 (ADLS Gen2)

- Azure SQL Database, Azure Blob Storage, Azure Key Vault, Azure Monitor / Log Analytics.

2. Programming & Databases :

- SQL (Advanced)

- Python

- PySpark

- Spark SQL, T-SQL.

3. Big Data Technologies :

- Apache Spark

- Delta Lake

- Data Lake Architecture

- Data Warehousing Concepts.

4. DevOps & Version Control :

- Azure DevOps, Git, CI/CD Pipelines.

5. Methodology :

- Agile/Scrum methodology.

Mandatory Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, or related field.

- 7+ years of experience in Data Engineering.

- Minimum 4+ years of hands-on experience with Microsoft Azure Data Platform.

- Strong experience with Azure Data Factory and Azure Databricks.

- Hands-on experience with Azure Synapse Analytics.

- Strong SQL and Python programming skills.

- Experience in building enterprise-scale ETL/ELT pipelines.

- Experience working with Azure Data Lake Storage.

- Good understanding of cloud architecture and distributed data processing.

Preferred Qualifications :

- Microsoft Azure Data Engineer Associate certification.

- Experience with Microsoft Fabric.

- Experience with Delta Lake and Lakehouse architecture.

- Knowledge of Power BI integration.

- Experience with Kafka or Event Hubs for streaming data.

- Exposure to Snowflake or other cloud data warehouses.

- Experience working in banking, healthcare, insurance, or retail domains.

Nice to Have :

- Microsoft Fabric

- Azure Functions

- Logic Apps

- Azure Event Hubs

- Azure Stream Analytics

- Docker

- Kubernetes

- Terraform, Purview, Power BI.

Soft Skills :

- Strong analytical and problem-solving skills.

- Excellent communication and stakeholder management.

- Ability to work independently and in a collaborative team environment.

- Strong documentation and presentation skills.

- Ability to mentor junior engineers and participate in technical design discussions.

Experience Requirements :

- Total IT Experience : 7+ Years

- Azure Data Engineering : 4+ Years

- Azure Data Factory : 4+ Years

- Azure Databricks & PySpark : 3+ Years

- Azure Synapse Analytics: 2+ Years

- SQL Development : 5+ Years

- Python Development : 3+ Years

Skills

Azure, Data Engineering, ETL, Data Pipeline, Big Data, SQL, Python, Data Warehousing, Azure Databricks, PySpark, Synapse Analytics