Lead/Principal Data Engineer - Data Engineering
First seen by Alion on Oct 26, 2026.
Role snapshot:
About Partex.AI:
Partex.AI works at the intersection of artificial intelligence and healthcare. We help leading life-sciences and healthcare organizations speed up medical research, diagnostics and therapeutic development through data-driven intelligence.
We are a fast-growing company with a global outlook and a focus on real-world impact. Our Data & AI team of 40+ engineers is scaling quickly, and we are building the leadership layer that will take it to the next stage.
About the role:
You will own the data platform that powers our AI products: large-scale crawling and ingestion, processing pipelines, storage, and the data behind our knowledge graph. You will lead a team of data engineers and be accountable for a platform that is reliable, scalable and cost-efficient.
This is a hands-on leadership role. You will set the architecture, stay close to the code, and work with data science and AI engineering teams so they get the data they need, when they need it.
Position:
Lead / Principal Data Engineer (1 opening)
Location:
Pune, India
Reporting to:
Head of Data & AI
Experience:
10+ years in data or backend engineering, including 3+ years leading engineers
Level:
Lead or Principal, based on depth of experience
Key responsibilities:
Technical ownership:
- Own the architecture of the data platform: ingestion, distributed crawling, batch and streaming pipelines, and our MongoDB and Elasticsearch data warehouse.
- Design and scale data pipelines that process large volumes of structured and unstructured healthcare and life-sciences data.
- Lead the data side of knowledge graph integration, including entity resolution, graph modelling and loading into graph databases.
- Set standards for data quality, lineage, monitoring, testing and documentation.
- Identify and fix performance, scalability and cost issues across the platform.
- Choose tools and infrastructure, and keep the stack current and maintainable.
Team leadership and delivery:
- Lead a team of data engineers: sprint planning, task allocation and day-to-day unblocking.
- Own delivery for your projects, including tracking progress, managing risks and giving status updates to stakeholders.
- Review code and designs, and raise the engineering bar of the team.
- Mentor engineers, run regular 1:1s and support their growth plans.
- Interview and hire for your team.
Business and clients:
- Design data architecture for client pitches and support technical due diligence.
- Work with data science and AI engineering leads to plan data needs for new products.
Must-have qualifications:
- 10+ years in data or backend engineering, with at least 3 years leading a team of engineers.
- Strong Python and solid computer science fundamentals: data structures, algorithms and system design.
- Experience designing and running large-scale distributed crawling and ingestion, for example with Scrapy.
- Hands-on experience with distributed processing such as Spark, Kafka or similar.
- Deep production expertise in MongoDB and Elasticsearch, the core of our data warehouse: data modelling, index and mapping design, query performance, sharding and scaling. Experience with at least one graph database such as Neo4j or ArangoDB.
- Experience with pipeline orchestration (Airflow or similar), Docker, Kubernetes and a major cloud platform.
- Proven ownership of delivery: planning, estimation and keeping stakeholders informed.
- Bachelor's or Master's degree in Computer Science or Engineering, or equivalent experience.
Nice-to-have qualifications:
- Experience with healthcare, life-sciences or biomedical data sources.
- Experience with lakehouse or warehouse platforms such as Databricks, Snowflake or BigQuery.
- Experience preparing data for ML and LLM workloads, including vector databases and embedding pipelines.
- Familiarity with data governance and compliance for sensitive data.
- Infrastructure-as-code experience with Terraform or similar.
Skills
Data Engineering, Python, MongoDB, ElasticSearch, Hadoop, Spark, Kafka