{"id":2836145,"url":"https://alion.io/job/calfus-lead-data-engineer","title":"Lead Data Engineer","company":{"id":3800027,"name":"Calfus","domain":"calfus.com","url":"https://alion.io/company/calfus","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":26000,"max_usd":45000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":16},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Lambda","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"NumPy","optional":false},{"name":"Pandas","optional":false},{"name":"PostgreSQL","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"SSIS","optional":false},{"name":"Rest API","optional":true}],"status":"live","first_seen_at":"2026-10-19T04:31:43Z","employer_posted_date":null,"last_verified_at":"2026-10-19T04:31:43Z","board_verified":false,"closed_at":null,"days_open":0,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":0},"description":"About the Role : \n\nAs a Lead Data Engineer, you'll own the design and delivery of our cloud data platforms on AWS and Azure. You'll architect lakehouse solutions built on Medallion Architecture, set the data modelling standards, and lead a team of data engineers delivering reliable, scalable pipelines. \n\nThis is a hands-on leadership role : you'll make the architecture decisions and still be close enough to the code to write SQL and PySpark yourself. With 8 - 12 years of experience, you've designed data platforms, not just built pipelines on them.\n\nWhat You'll Do : \n\nData Architecture & Platform Design : \n\n- Design scalable data lake, lakehouse, and warehouse architectures on AWS and Azure that meet business requirements.\n\n- Implement Medallion Architecture (bronze, silver, gold) using Databricks and Delta Lake.\n\n- Work with stakeholders to gather requirements and translate them into technical specifications and architecture designs.\n\nData Integration & Pipelines : \n\n- Build and oversee ETL/ELT pipelines using Azure Data Factory, Airflow, SSIS, or equivalent tools.\n\n- Use Azure Databricks and PySpark for large-scale data processing.\n\n- Own pipeline reliability - orchestration, monitoring, failure handling, and automation.\n\nData Modelling : \n\n- Design and maintain dimensional and relational data models (star, snowflake) for analytical workloads.\n\n- Own schema design decisions and modelling standards across the platform.\n\nDatabase Engineering & Performance : \n\n- Write complex SQL queries, stored procedures, and transformations.\n\n- Optimise pipelines, queries, and Spark jobs for performance, scalability, reliability, and cost.\n\nData Quality & Governance : \n\n- Implement data quality checks, validation, and governance best practices to ensure accurate and compliant data.\n\nTeam Leadership : \n\n- Lead, mentor, and guide data engineers, review code, and build a culture of continuous learning and improvement.\n\nWhat You'll Need - Must Have : \n\n- 8 - 12 years in data engineering, including experience leading data engineering teams.\n\n- Strong AWS or Azure experience (S3, Lambda, Glue / ADF, ADLS, Databricks) demonstrated through delivered projects.\n\n- Medallion Architecture - you've designed and implemented bronze/silver/gold layers on Databricks or Delta Lake.\n\n- Data Architecture - you've designed end-to-end data platforms, not only built pipelines on them.\n\n- Data Modelling - you've built dimensional and relational models, not only queried them.\n\n- Advanced SQL - SQL Server, Postgres, Snowflake, or Redshift, including stored procedures and query tuning.\n\n- ETL/ELT and orchestration tools such as SSIS, Azure Data Factory, or Airflow.\n\n- Python for data engineering - Pandas, NumPy, PySpark, scripting, and automation.\n\n- Experience with databases like Snowflake, Postgres, Redshift, and MongoDB.\n\n- Familiarity with data formats like JSON, CSV, Parquet, and Pickle.\n\n- Version control (Git), CI/CD, and code quality practices for data engineering projects.\n\n- Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field.\n\nGood to Have : \n\n- Hands-on experience across both AWS and Azure.\n\n- Azure SDK.\n\n- Working with REST APIs and web scraping.\n\n- Unity Catalog or other data governance and cataloguing tools.\n\nBenefits : \n\nAt Calfus, we value our employees and offer a strong benefits package. This includes medical, group, and parental insurance, coupled with gratuity and provident fund options. Further, we support employee wellness and provide birthday leave as a valued benefit.\nSkills\nData Engineering, ETL, Azure Data Factory, Apache Airflow, SQL, PySpark, Data Modeling, Data Integration, Data Quality, Data Governance, Python","description_format":"text","description_chars":3706,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Agents","AI Consulting & Integration"],"lifecycle":[{"event":"open","at":"2026-10-19T06:29:30Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 1","filings_12m":1,"filings_prev_12m":1,"green_card_filings_12m":0,"median_offered_wage_usd":152500,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":0}],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":26,"reasons":["seen:0","win:early"],"computed_at":"2026-10-20T02:36:17Z"},"pay":null,"html_url":"https://alion.io/job/calfus-lead-data-engineer","json_url":"https://alion.io/job/calfus-lead-data-engineer.json","meta":{"generated_at":"2026-10-20T02:36:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2947,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-21T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3800027},"rest":"https://alion.io/mcp/rest/get_company?id=3800027"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcalfus-lead-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcalfus-lead-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcalfus-lead-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/calfus-lead-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcalfus-lead-data-engineer"}]}