Data Engineer

Functional Reporting :

Functional reporting : Group leader

Administrative reporting : DPM

LOCATION : Gurugram, Haryana

Educational Qualification :

- B.E / B. Tech in Computer Science/IT/ or Masters in Computer Application(MCA)

- Minimum 60 % marks

- Minimum 3 Years work experience in Big Data/Cloud Computing/ Data Engineering/ Analytics related projects

Desirable :

- Certification in Big Data/Data Science/Cloud Computing or equivalent

Experience :

- Minimum : 5 Year(s), Maximum : 8 Year(s)

Job Role & Responsibilities :

- Data Engineering & Quality Management : Design, implementation, and management of data platform, data pipelines to help fulfill data access and aggregation functionality for users in cloud environment.

- Develop and Maintain Data Pipelines : Design, build, and manage scalable data pipelines for data extraction, transformation, and loading (ETL).

- Data Integration : Integrate data from various sources, ensuring data quality and consistency.

- Optimize Data Systems : Identify and implement process improvements to enhance data reliability, efficiency, and quality.

- Collaborate with Teams : Work with data scientists, analysts, and other stakeholders to support their data infrastructure needs.

- Monitor and Troubleshoot : Monitor data pipelines and systems, troubleshoot issues, and ensure data integrity.

- Documentation : Maintain comprehensive documentation of data processes and systems.

Technical/ Functional Competencies :

- Experience : 3 - 5 years of experience in data engineering or a related field.

- Technical Skills : Proficiency in Py-Spark (Must Have), SQL, Python, and data platform like Databricks.

- Storage Management : Experience in storage mechanism in AWS S3, Layers of Medallion Architecture, OTF like Delta, iceberg etc., Hand-Ons in Data schema like Star, snowflake etc.

- Good knowledge next generation ETL/ELT tools and data consolidation frameworks that enable capabilities within the data ecosystem.

- Ability to work independently or within a team, proactively, in a fast-paced AGILE-SCRUM environment.

Desired but not essential :

- Experience in IoT time series data domain.

- Exposure to automotive data.

Behavioral competencies :

- Effective team player.

- Study industry trends (global and domestic) and apply it in work-area.

- Co-ordinate department activities and represent department / MSIL in forums.

- Good communication and presentation skills.

Skills

Data Engineering, Data Quality, Data Management, Data Pipeline, ETL, Data Integration, Python, SQL, PySpark, Medallion Architecture