Cloud Engineer/ Support Engineer - Cloud

Role

Summary:

This

job involves creating data pipelines using a configurable framework

(developed using Databricks / pyspark), which are used to load data from

multiple sources to Google Cloud. It also involves extensive collaboration

with other team members (India & USA) to follow a plan, execute

workflows, ensure data load accuracy and debug in case of issues.

Essential

Responsibilities

Design, develop, and implement robust, high-performance data pipelines using

Databricks/PySpark to meet diverse data integration requirements. Understand

data and its relation / significance from business perspective

Leverage extensive SQL expertise for complex data manipulation, analysis, and

query optimization.

Experience in Google Cloud services like Dataproc, BigQuery, Airflow, etc.

Create custom scripts in python / pyspark

Minor development and enhancement activities as needed to handle new / custom

requirements

Validate data between source and target & highlight issues and rerun jobs

as needed

Lead other developers, review their code and ensure quality & optimal

code is being delivered

Provide regular updates on the tasks, status and risks to project manager

The

experience we are looking to add to our team

Required

Bachelor’s degree from a reputed university

7 to 12 years overall experience

Excellent SQL / data knowledge (6+ years) in creating complex SQL queries,

Stored Procedures etc.

Experience in developing enterprise grade ETL applications (any tool)

Experience in Google BigQuery or any cloud database

Experience in Google Cloud services like Managed service for Apache

Spark(Dataproc), Airflow etc.

Working knowledge in Pyspark / Python

Good communication and positive attitude

Attitude to take ownership and drive tasks without follow-ups

Good

to have

Experience in Big data technologies like Hadoop, Hive, etc.

Knowledge in Iceberg, Starburst, DBT

Knowledge in US Healthcare domain

Knowledge in Agile