Data Engineer

Data Engineer

As a Principal Data

Engineer, your responsibilities will include:

• Design and build data pipelines

  to process terabytes of data

• Orchestrate in Airflow the data

  tasks to run on Kubernetes/Hadoop for the ingestion, processing and

  cleaning of data.

• Create Docker images for

  various applications and deploy them on Kubernetes

• Design and build best in class

  processes to clean and standardize data.

• Troubleshoot production issues in

  our Elastic Environment

• Tuning and optimizing data

  processes

· Advancing the team’s DataOps culture (CI/CD,

Orchestration, Testing, Monitoring) and building out standard development

patterns

• Drive innovation by

  testing new technology and approaches to continually advance the

  capability of the data engineering function.

• Drive efficiencies in

  current engineering processes via standardization and migration of

  existing on-premise processes to the cloud

• Ensuring Data

  Quality - building best in class data quality monitoring that

  ensure that all data products exceed customer expectations.

Required Qualifications:

• Computer Science bachelor’s

  degree or similar.

• Good understanding of Data Modelling techniques i.e.

  DataVault, Kimble Star

• Excellent understanding of Column-Store RDBMS

  (DataBricks, Snowflake, Redshift, Vertica, Clickhouse)

• Good experience handling real-time, near real-time

  and batch data ingestions

• Hands on experience on the

  following technologies:

• Developing processes in Spark
• Writing complex SQL queries f
• Building ETL/data pipelines
• Exposure to Kubernetes and

  Linux containers (i.e. Docker)

• Related/complementary open

  source software platforms and languages (e.g. Scala, Python, Java, Linux)

• Proven track

  record of designing effective data strategies and leveraging modern data

  architectures that resulted in business value

• Experience

  building cloud-native data pipelines on either AWS, Azure or GCP,

  following best practices in cloud deployments

·Strong DataOps experience (CI/CD,

Orchestration, Testing, Monitoring)

• Strong

  experience leading and developing data engineering teams

·Demonstrated effective interpersonal,

influence, collaboration and listening skills

·Strong stakeholder management skills

·Excellent time management, organizational and

prioritization skills with ability to balance multiple priorities.

Preferred Qualifications:

·Experience with data tokenization and different techniques and

tools i.e. DataVant, Protegrity

·Experience with Azure Data Factory, Databricks and Snowflake

• Experience

  with Apache Spark and related Big Data stack and technologies, PySpark

  Scala

·Experience working with Apache Kafka, building appropriate

producer/consumer apps

·Experience working with Kubernetes and Docker, and knowledgeable

about cloud infrastructure automation and management (e.g., Terraform)

·Experience working in projects with agile/scrum methodologies

·Familiarity with production quality ML and/or AI model development

and deployment.

·Healthcare industry knowledge and experience with exposure to EDI,

HIPAA, HL7 and FHIR integration standards