{"id":4196231,"url":"https://alion.io/job/bidgely-senior-data-engineer","title":"Senior Data Engineer","company":{"id":11694,"name":"Bidgely","domain":"bidgely.com","url":"https://alion.io/company/bidgely","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Airflow","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Cortex","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"LangChain","optional":false},{"name":"LangGraph","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Agile","optional":true},{"name":"dbt","optional":true},{"name":"Looker","optional":true},{"name":"Prefect","optional":true},{"name":"Prometheus","optional":true},{"name":"Tableau","optional":true}],"status":"live","first_seen_at":"2026-11-12T17:28:49Z","employer_posted_date":null,"last_verified_at":"2026-11-12T17:28:49Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"Roles & Responsibilities\n\nPlay a key hands-on role in building and running central data platform features on Databricks and Snowflake, supporting the architects and managers who own the design and delivery. \nDesign, build, and maintain scalable batch and streaming ELT/ETL data pipelines and large-scale datasets using Apache Spark with Python or Scala, Delta, Kafka and Airflow. \nBuild AI agents and agentic data products on Databricks (for example Genie), Snowflake (for example Cortex) and LangGraph, and expose governed data to agents through MCP tools built with FastMCP. \nImplement automated data quality checks, end-to-end lineage tracking and RBAC security policies for the datasets you build, following the team's data governance framework. \nMonitor and optimize Databricks and Snowflake compute and storage costs, and work within the team's budget and spend-alert policies. \nWork closely with product managers, BI engineers, and data scientists to translate business requirements into scalable, well-modelled analytical solutions. \nSupport data modelling initiatives using dimensional (Kimball) and normalized (3NF) techniques to serve AWB reporting and analytics needs. \nContribute to CI/CD pipeline development for data workflows, including version control, automated testing, and deployment standards. \nReview code and share best practices on Databricks, Snowflake and data quality with the team. This is an individual contributor role with no direct reports. \nTranslate architectural designs into high-quality, production-ready code by applying software design principles, design patterns, SOLID principles, and clean code practices. Build modular, reusable, loosely coupled, and testable components that support long-term scalability and maintainability. \n\nWhat You'll Deliver In Your First 12 Months\n\nDeliver core data platform features end to end in production on Databricks and Snowflake, working with the architects and managers who own them. \nDeliver agent-ready datasets and tools that LangGraph agents and MCP tools use in production. \nHave data quality checks, lineage and RBAC in place for every dataset you build. \nBring the compute and storage cost of the pipelines you build within agreed budgets. \n\nWhat You'll Bring\n\n5+ years of data engineering experience building central data platforms, large-scale pipelines, and production datasets; strong, hands-on proficiency in both Databricks and Snowflake is strictly required. \nStrong proficiency in Apache Spark using Python or Scala on Databricks, and in SQL on both Databricks and Snowflake, for core data processing workloads. Strong coding skills are required. This is a hands-on engineering role. \nHands-on experience with Delta Lake as the primary table format, including MERGE, schema evolution, OPTIMIZE, VACUUM and performance tuning. \nHands-on experience building streaming pipelines with Kafka and Spark Structured Streaming, including duplicates, late data, checkpoints and delivery guarantees. \nProduction experience orchestrating batch and streaming workflows with Apache Airflow (mandatory), including retries, backfills, dependencies and alerting. \nHands-on experience building AI agents with LangGraph (or LangChain) and with the agent features of Databricks and Snowflake, such as Genie and Cortex. Uses AI code-generation tools in day-to-day engineering work. \nSolid understanding of data governance concepts including data lineage, quality monitoring, regulatory compliance, RBAC, and governance frameworks. \nExperience building production-grade data pipelines on cloud platforms (AWS, Azure, or GCP) with a focus on reliability and observability. \nGood understanding of data modelling concepts including Kimball dimensional modelling, star/snowflake schemas, and 3NF normalization. \nFamiliarity with CI/CD pipelines, Git-based workflows, and applying DevOps practices to data engineering projects. \nStrong communication and stakeholder management skills with an ownership mindset and ability to work independently across time zones. \n\nBonus Skills / Nice To Have\n\nExposure to Databricks and Snowflake Native Applications, data clean rooms, or Marketplace integrations. \nExperience implementing, testing and documenting dbt models for data reliability, lineage tracking and governance. \nExperience with other orchestration tools such as Prefect or dbt Cloud. \nExperience supporting BI/Analytics teams through semantic layer development (e.g., Looker, Tableau, or similar tools). \nExperience working in Agile/product-based development environments with sprint planning, backlog management, and iterative delivery. \nDatabricks and Snowflake cost optimization experience: compute usage analysis, query profiling, clustering and partitioning design, materialized views, search optimization, resource monitors, budgets and spend alerts. \nBackground in the energy, utilities, or IoT analytics domain is a strong differentiator. \nExposure to MLOps concepts and production deployment workflows for data science models integrated with Databricks and Snowflake.","description_format":"text","description_chars":5046,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Energy Efficiency","Energy Management"],"lifecycle":[{"event":"open","at":"2026-11-12T20:32:35Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":30,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-11-14T05:58:10Z"},"pay":null,"html_url":"https://alion.io/job/bidgely-senior-data-engineer","json_url":"https://alion.io/job/bidgely-senior-data-engineer.json","meta":{"generated_at":"2026-11-14T16:17:37Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":18366,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":11694},"rest":"https://alion.io/mcp/rest/get_company?id=11694"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbidgely-senior-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbidgely-senior-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbidgely-senior-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/bidgely-senior-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbidgely-senior-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4196231}},{"tool":"search_jobs","arguments":{"company":"bidgely.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4196231},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}