Data Engineer

Job Summary

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and data platforms that support analytics, reporting, and machine learning initiatives. The ideal candidate will have experience working with large-scale data processing, cloud-based data solutions, ETL/ELT frameworks, and modern data warehousing technologies.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources.
• Build and optimize data architectures, data lakes, and data warehouses for structured and unstructured data.
• Develop and manage batch and real-time data processing pipelines.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize SQL queries, data models, and pipeline performance to ensure efficient data processing.
• Ensure data quality, integrity, security, and governance across all data platforms.
• Implement data validation, monitoring, and troubleshooting processes.
• Work with cloud-based data services and modern data engineering tools.
• Develop and maintain technical documentation for data pipelines, workflows, and architecture.
• Support data migration and integration projects involving multiple enterprise systems.
• Participate in code reviews and follow best practices for data engineering and DevOps.

Required Skills

• Strong programming experience in Python, SQL, and/or Scala.
• Hands-on experience with ETL/ELT tools and frameworks.
• Strong knowledge of relational databases and data modeling concepts.
• Experience with modern data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics.
• Experience building data pipelines using Apache Spark, PySpark, or similar distributed processing frameworks.
• Knowledge of workflow orchestration tools such as Apache Airflow or Azure Data Factory.
• Experience working with cloud platforms such as Microsoft Azure, AWS, or Google Cloud Platform (GCP).
• Familiarity with data lakes, data governance, and data quality best practices.
• Experience with Git and version control.
• Strong analytical, troubleshooting, and problem-solving skills.

Preferred Skills

• Experience with streaming technologies such as Apache Kafka, Azure Event Hubs, or Amazon Kinesis.
• Knowledge of Databricks and Delta Lake.
• Experience with Infrastructure as Code (Terraform, ARM, or CloudFormation).
• Understanding of DevOps and CI/CD pipelines for data engineering.
• Exposure to Machine Learning data pipelines and feature engineering.
• Experience with REST APIs and third-party data integrations.

Technical Skills

• Python
• SQL
• PySpark / Apache Spark
• Apache Airflow
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Snowflake
• Databricks
• Azure Data Lake Storage (ADLS)
• AWS S3 / Amazon Redshift
• Google BigQuery
• Apache Kafka
• Git
• Docker (Preferred)

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
• Relevant certifications in Azure, AWS, GCP, or Snowflake are preferred.

Preferred Experience

• 5-8 years of experience in Data Engineering.
• Experience designing and implementing enterprise-scale data pipelines.
• Hands-on experience with cloud-based data platforms and modern data architectures.
• Experience working in Agile/Scrum environments.

Nice to Have

• Experience supporting AI/ML workloads and data pipelines.
• Knowledge of Data Mesh, Data Fabric, or Lakehouse architecture.
• Familiarity with Power BI, Tableau, or other BI tools.
• Experience with data governance frameworks and metadata management tools.
• Exposure to Generative AI data preparation and vector databases.

What We Offer

• Opportunity to work on enterprise-scale cloud and data transformation projects.
• Exposure to modern data engineering technologies and cloud platforms.
• Collaborative and innovation-driven work environment.
• Career growth and learning opportunities.
• Competitive compensation and comprehensive benefits.
• Flexible work arrangements (Remote/Hybrid, based on business needs).