Senior Data Engineer (Databricks)

Job Requirements

Mandatory Skills

Spark,Databricks Engineer,Kafka,Stream Processing,SQL

Development,Python,Snowflake

Job Description

Architecture & Strategy: End-to-end design of scalable, secure, and highly available

data architectures leveraging modern cloud data ecosystems (Databricks and

Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable

streaming and batch data pipelines (ETL/ELT) to process complex, large-scale

datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform

components natively within the AWS ecosystem, ensuring tight integration with core

security, IAM, and networking protocols.

API Ingestion & Orchestration: Design and implement robust data ingestion

frameworks leveraging Databricks APIs and external REST/GraphQL APIs for

automated workflows, platform orchestration, and data delivery.

Real-time Processing: Design and implement robust frameworks for real-time data

ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling: Harmonize old school relational data warehousing patterns

(Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern

paradigms.

Technical Leadership: Act as a core problem-solver for complex data bottlenecks,

provide technical governance, and mentor engineering teams on data best practices.

AI Integration: Collaborate with Data Science and AI teams to architect data layers that

seamlessly support LLMs, Machine Learning pipelines, and advanced analytics

solutions.

Education Qualificaiton

B Tech/ M Tech

Roles & Responsibilities

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data

architectures leveraging modern cloud data ecosystems (Databricks and Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming

and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components

natively within the AWS ecosystem, ensuring tight integration with core security, IAM, and

networking protocols.

API Ingestion & Orchestration: Design and implement robust data ingestion frameworks

leveragin

Work Experience

7-10Years