Senior Data Engineer
Smart Apply
Similar jobs
Location
In office (Bengaluru)
Seniority
Senior · 5+ years exp
Employment
Full-Time
First seen by Alion on Nov 12, 2026.
Overview
Company
Impact
Profile match
Bidgely analyzes household energy data consumption patterns to the appliance level without using any plug-level monitors. The company has residential customers on six continents and works with leading utilities around the world.
Backed by Future Energy Ventures GmbH, Accurantllc
Roles & Responsibilities
- Play a key hands-on role in building and running central data platform features on Databricks and Snowflake, supporting the architects and managers who own the design and delivery.
- Design, build, and maintain scalable batch and streaming ELT/ETL data pipelines and large-scale datasets using Apache Spark with Python or Scala, Delta, Kafka and Airflow.
- Build AI agents and agentic data products on Databricks (for example Genie), Snowflake (for example Cortex) and LangGraph, and expose governed data to agents through MCP tools built with FastMCP.
- Implement automated data quality checks, end-to-end lineage tracking and RBAC security policies for the datasets you build, following the team's data governance framework.
- Monitor and optimize Databricks and Snowflake compute and storage costs, and work within the team's budget and spend-alert policies.
- Work closely with product managers, BI engineers, and data scientists to translate business requirements into scalable, well-modelled analytical solutions.
- Support data modelling initiatives using dimensional (Kimball) and normalized (3NF) techniques to serve AWB reporting and analytics needs.
- Contribute to CI/CD pipeline development for data workflows, including version control, automated testing, and deployment standards.
- Review code and share best practices on Databricks, Snowflake and data quality with the team. This is an individual contributor role with no direct reports.
- Translate architectural designs into high-quality, production-ready code by applying software design principles, design patterns, SOLID principles, and clean code practices. Build modular, reusable, loosely coupled, and testable components that support long-term scalability and maintainability.
- Deliver core data platform features end to end in production on Databricks and Snowflake, working with the architects and managers who own them.
- Deliver agent-ready datasets and tools that LangGraph agents and MCP tools use in production.
- Have data quality checks, lineage and RBAC in place for every dataset you build.
- Bring the compute and storage cost of the pipelines you build within agreed budgets.
- 5+ years of data engineering experience building central data platforms, large-scale pipelines, and production datasets; strong, hands-on proficiency in both Databricks and Snowflake is strictly required.
- Strong proficiency in Apache Spark using Python or Scala on Databricks, and in SQL on both Databricks and Snowflake, for core data processing workloads. Strong coding skills are required. This is a hands-on engineering role.
- Hands-on experience with Delta Lake as the primary table format, including MERGE, schema evolution, OPTIMIZE, VACUUM and performance tuning.
- Hands-on experience building streaming pipelines with Kafka and Spark Structured Streaming, including duplicates, late data, checkpoints and delivery guarantees.
- Production experience orchestrating batch and streaming workflows with Apache Airflow (mandatory), including retries, backfills, dependencies and alerting.
- Hands-on experience building AI agents with LangGraph (or LangChain) and with the agent features of Databricks and Snowflake, such as Genie and Cortex. Uses AI code-generation tools in day-to-day engineering work.
- Solid understanding of data governance concepts including data lineage, quality monitoring, regulatory compliance, RBAC, and governance frameworks.
- Experience building production-grade data pipelines on cloud platforms (AWS, Azure, or GCP) with a focus on reliability and observability.
- Good understanding of data modelling concepts including Kimball dimensional modelling, star/snowflake schemas, and 3NF normalization.
- Familiarity with CI/CD pipelines, Git-based workflows, and applying DevOps practices to data engineering projects.
- Strong communication and stakeholder management skills with an ownership mindset and ability to work independently across time zones.
- Exposure to Databricks and Snowflake Native Applications, data clean rooms, or Marketplace integrations.
- Experience implementing, testing and documenting dbt models for data reliability, lineage tracking and governance.
- Experience with other orchestration tools such as Prefect or dbt Cloud.
- Experience supporting BI/Analytics teams through semantic layer development (e.g., Looker, Tableau, or similar tools).
- Experience working in Agile/product-based development environments with sprint planning, backlog management, and iterative delivery.
- Databricks and Snowflake cost optimization experience: compute usage analysis, query profiling, clustering and partitioning design, materialized views, search optimization, resource monitors, budgets and spend alerts.
- Background in the energy, utilities, or IoT analytics domain is a strong differentiator.
- Exposure to MLOps concepts and production deployment workflows for data science models integrated with Databricks and Snowflake.
Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,285,881 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Free forever. No card. Under a minute.
Your match
How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.
Recommended for you based on this role
Data Science
Similar stack
Same company
Bengaluru
up to $27k per year · Remote (India) · Full-Time · 4+ years exp · Bengaluru
Python
SQL
Python
pySpark
Databases
Snowflake
Databricks
AI/ML
Hadoop
Spark
DevOps
Azure
CI/CD
AWS
Analytics
ETL/ELT
Management
Agile
Apply
In office · 2+ years exp · Bachelor's Degree · Pune
Python
SQL
Python
pySpark
Databases
Databricks
Delta Lake
Azure SQL Database
AI/ML
Copilot
Spark
DevOps
Azure
CI/CD
Git
Analytics
ETL/ELT
Informatica
Azure Data Factory
Management
Agile
Oracle ERP
Apply
Apply
Payments Data Scientist Associate
8 hours ago
In office · 3+ years exp · Mumbai
Python
SQL
Analytics
Tableau
Apply
Senior Data Scientist - Machine Learning
11 hours ago
In office · 6+ years exp · Kolkata · Delhi
Python
SQL
Python
FastAPI
Databases
PostgreSQL
AI/ML
LangGraph
LangChain
Embeddings
Prompt Engineering
Function Calling
AI Agents
LLM
RAG
Hallucination
Time Series
LLM Guardrails
Agentic Workflows
Multi-Agent Systems
Machine Learning
DevOps
GCP
Azure
CI/CD
Git
AWS
Docker
Apply
Senior Data Engineer - Azure/Databricks
5 hours ago
In office · 4+ years exp · Gurgaon · Bengaluru · Mumbai
Python
SQL
Databases
Databricks
AI/ML
Spark
DevOps
GCP
Azure
AWS
Apply
In office · Full-Time · 1+ year exp · Bachelor's Degree · Bengaluru
AI/ML
Instructor
Apply
SAP Fiori Consultant
1 hour ago
In office · 3+ years exp · Bengaluru
JavaScript
ABAP
ABAP
SAP Fiori
SAP UI5
Apply
Manager Product Development
1 hour ago
In office · 2+ years exp · Bengaluru
Python
JavaScript
TypeScript
Node JS
Python
FastAPI
DevOps
AWS
Apply
DevOps Engineer
1 hour ago
In office · 3+ years exp · Bengaluru · Chennai · Pune
Python
Java
DevOps
Splunk
Terraform
Ansible
Prometheus
CI/CD
GitOps
AWS
Docker
Kubernetes
Grafana
Amazon EKS
Cybersecurity
ISO 27001
SOC 2
Apply
This is one of many
2,285,881 more open roles from verified company boards, updated every day.