{"id":3079372,"url":"https://alion.io/job/inferyx-analytics-lead-data-engineer-2","title":"Lead Data Engineer","company":{"id":3812593,"name":"Inferyx Analytics","domain":"inferyx.com","url":"https://alion.io/company/inferyx-analytics","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":28000,"max_usd":60000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon Redshift","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Hadoop","optional":false},{"name":"Java","optional":false},{"name":"MySQL","optional":false},{"name":"PostgreSQL","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-10-22T11:35:15Z","employer_posted_date":null,"last_verified_at":"2026-10-22T11:35:15Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"We are looking for a Lead Data Engineer who is responsible for the design and development of scalable data pipelines and integrations to support continual increases in data volume and complexity. Work with analytics and business teams to understand their needs, create pipelines, and improve data models that feed BI and visualisation tools.\n\nThe candidate will have responsibilities across the following functions: \n\nData Pipeline Development: \nDesign, develop, and maintain scalable batch and streaming data pipelines using Apache Spark (PySpark/Scala) and Databricks.\nBuild end-to-end ETL/ELT workflows for ingesting, transforming, and validating data from diverse source systems while ensuring data accuracy, reliability, and performance.\n\nData Modelling and Analytics Enablement: \nDesign and maintain efficient data models, schemas, and curated datasets that support business analytics, reporting, and visualisation tools.\nOptimise data structures for performance, scalability, and cost across lakehouse and data warehouse platforms.\n\nData Integration: \nIntegrate data from multiple internal and external sources, including relational databases, APIs, flat files, and streaming sources.\nEnsure seamless and reliable data movement across cloud platforms, data lakes, and analytics systems.\n\nPerformance Optimisation: \nIdentify and resolve performance bottlenecks in Spark jobs, Databricks workloads, and data storage layers.\nTune Spark configurations, optimise queries, and improve pipeline efficiency to support large-scale data processing.\n\nData Quality and Governance: \nImplement data quality checks, validation rules, and governance standards to ensure trustworthy data.\nMonitor data quality metrics and proactively address data issues in collaboration with stakeholders.\n\nCollaboration and Stakeholder Engagement: \nWork closely with data analysts, data scientists, and business teams to understand requirements and deliver data solutions aligned with business objectives.\nPartner with platform and cloud teams to ensure architectural consistency and best practices.\n\nDocumentation and Best Practices: \nDocument data pipelines, data models, and technical designs.\nFollow best practices for software development, version control, CI/CD, and deployment in distributed data environments.\n\nContinuous Improvement: \nStay current with emerging data engineering technologies, Spark and Databricks enhancements, and cloud data platform innovations.\nDrive automation and process improvements to increase reliability, scalability, and developer productivity.\n\nRequirements: \nBachelor's degree in Computer Science, Engineering, or related field.\n8+ years of experience in data engineering or related roles.\nProficiency in programming languages such as Python, Java, or Scala.\nStrong SQL skills and experience with relational databases (e. g., MySQL, PostgreSQL).\nExperience with data warehousing concepts and technologies (e. g., Snowflake, Redshift).\nFamiliarity with big data processing frameworks (e. g., Apache Spark, Hadoop).\nHands-on experience with ETL tools and data integration platforms.\nKnowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform.\nUnderstanding of data modelling principles and data warehousing design patterns.\nExcellent problem-solving skills and attention to detail.\nStrong communication and collaboration.\nSkills\ndata engineering, databricks, java, pyspark, python, scala, spark","description_format":"text","description_chars":3421,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Analytics & BI Consulting","Data Engineering & Migration Services"],"lifecycle":[{"event":"open","at":"2026-10-22T11:35:15Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":4,"expected_fill_days":27,"reasons":["seen:4","win:early"],"computed_at":"2026-10-27T05:46:45Z"},"pay":null,"html_url":"https://alion.io/job/inferyx-analytics-lead-data-engineer-2","json_url":"https://alion.io/job/inferyx-analytics-lead-data-engineer-2.json","meta":{"generated_at":"2026-10-27T14:07:41Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2542,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-28T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3812593},"rest":"https://alion.io/mcp/rest/get_company?id=3812593"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Finferyx-analytics-lead-data-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Finferyx-analytics-lead-data-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Finferyx-analytics-lead-data-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/inferyx-analytics-lead-data-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Finferyx-analytics-lead-data-engineer-2"}]}