Senior Data Engineer

Key Responsibilities

• Build and test data pipelines on modern data platforms

• Support data engineering, ingestion, transformation and consumption activities

• Develop new pipelines and data products, and automation

• Enhance existing pipelines and data products, and provide support

• Maintain all the data pipelines and data products of these Solutions

• Design conceptual, logical, and physical data models

• Design data lake to support analytical and reporting needs

• Provide technical services like development, data preparation, data transformation, data

encryption, data masking, data optimization and data testing preparation

• Adhere to the standards and guidelines defined by the Authority for devops processes, provide

deployment plan, checklist and assist in production implementation/ roll out

• Provide support and integration with capability on latest AI tool stack to expedite the

development, testing and deployment process

 Required Skills

• Python

• PySpark

• Snowflake

• Databricks

• Cloudera

• Migration from Cloudera to Snowflake or Databricks with meta data-driven design

 Required Experience

• Minimum 5+ years of experience in the software used by the Solution

• Good understanding of data warehouse implementations

• Hands-on experience in big data engineering jobs using Python, PySpark to build and test data

pipelines on modern data platforms such as Snowflake, Databricks and Cloudera

Skills

Snowflake Cloud Data Warehouse, Cloudera, Production Technical Services, PySpark, Provides Support, Data Modelling, Conceptual Design, Logic, Hadoop Database, Data Pipeline, Pipeline Development, Data Transformation, Databricks, Data Engineering, Data Lake, Python, Snowflake Computing, Content Ingest Operations, Data Preparation