ADF Databricks Pyspark Python Terraform

TCS Hiring for ADF Databricks Pyspark Python Terraform

Experience Range: - 06 To 09 Years (Mandatory)

(Note: Candidates below 6 years of IT experience shall not be considered)

JOB LOCATION: Hyderabad, Pune & Indore

JOB DESCRIPTION

Key Responsibilities:

• Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
• Develop batch and near real-time data processing solutions using PySpark and Python.
• Build and maintain enterprise data lakes and data warehouse solutions.
• Implement data ingestion frameworks for structured, semi-structured, and unstructured data.
• Develop reusable data transformation and data quality frameworks.
• Provision and manage Azure infrastructure using Terraform.
• Optimize Databricks jobs, clusters, and Spark workloads for performance and cost efficiency.
• Implement CI/CD pipelines and deployment automation.
• Collaborate with architects, data analysts, and business stakeholders to gather requirements.
• Perform data validation, monitoring, troubleshooting, and production support.
• Ensure compliance with security, governance, and data management standards.

Desired Candidate Profile:

• Strong hands-on experience in Azure Databricks, ADF, PySpark, Python, and Terraform.
• Experience building scalable cloud-native data platforms.
• Strong understanding of data lake, data warehouse, and ETL concepts.
• Expertise in Spark performance tuning and optimization.
• Excellent problem-solving and analytical skills.
• Experience working in Agile/Scrum environments.
• Strong communication and stakeholder management skills.

Mandatory Technical Skills:

1) Azure Services: Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS), Azure Key Vault, Azure Monitor, Azure DevOps, Azure SQL Database

2) Programming & Data Engineering: Python, PySpark, SQL, Apache Spark, Data Modeling, ETL/ELT Development, Data Warehousing Concepts

3) Infrastructure as Code: Terraform, Azure Resource Management, Infrastructure Automation

4) DevOps & Version Control: Azure DevOps, Git, CI/CD Pipelines

5) Databases: Azure SQL, SQL Server, PostgreSQL, Cosmos DB (Preferred)

Minimum Qualification:

15 years of Full Time Education

Interested candidates kindly share your updated resume to [HIDDEN TEXT] along with filling the following details to maximize your chances of getting calls from TCS. The details are as follows:

Section 1: Basic Details:

• Candidate Name:
• Current Location (City):
• Preferred Location (City):

Section 2: Educational Details:

• Highest Regular Full‑Time Qualification
• (Diploma / Graduation / Post Graduation / Others)
• 10th Completion Year
• 12th / Intermediate Completion Year
• Graduation Completion Year
• Post‑Graduation Completion Year (if applicable)

Section 3: Work Experience:

• Current Company Name
• Total Experience (in years)
• Number of Companies Worked
• Employment Timeline
• (Example: Company A – 2021 to Present, Company B – 2019 to 2021)

Section 4: Compensation Details:

• Current CTC (in LPA)
• Expected CTC (in LPA)
• Notice Period
• (Immediate / 15 Days / 30 Days / 60+ Days)

Section 5: Documentation & Compliance (Yes / No):

• Valid ID Proof Available (Aadhar & Pan Card): (Yes/No)
• Educational Documents Available: (Yes/No)
• Employment Documents Available: (Yes/No)
• UAN Number Available: (Yes/No)