Lead Data Engineer
First seen by Alion on Aug 31, 2026.
About the Role:
We are looking for a Lead Data Specialist to own and manage the entire data lifecycle for our algorithmic trading systems. This role is critical to the reliability of our research, backtesting, and live trading environment. You will lead data engineering, ETL pipelines, data quality processes, and vendor integrations. This is a high-ownership role working directly with quants, traders, and tech leads.
Key Responsibilities:
1. Data Pipeline Ownership:
- Design, build, and maintain scalable ETL pipelines for tick, OHLC, fundamentals, and alternative data.
- Develop automated ingestion systems for both real-time and historical market data.
- Ensure fault-tolerant, high-availability data workflows.
2. Data Quality & Integrity:
- Detect and fix data anomalies: stale ticks, bad prints, missing data, timestamp issues.
- Implement automated QA, validation checks, and reconciliation workflows.
- Own daily data quality reporting.
3. Data Architecture & Storage:
- Manage storage layers.
- Optimize data retrieval for quants and live trading systems.
- Implement data versioning and point-in-time correctness for backtesting.
4. Vendor & Exchange Data Management:
- Integrate and manage feeds from Bloomberg, Refinitiv, FactSet, Tick data providers, NSE/BSE, global exchanges.
- Be primary contact for vendor issue resolution.
- Ensure compliance with vendor licensing and data usage policies.
5. Cross-Team Collaboration:
- Work closely with quant researchers to understand data needs and create feature-ready datasets.
- Support engineering teams in live feed integration, API access, and latency monitoring.
- Work with product and compliance on data governance and documentation.
6. Leadership & Mentorship:
- Lead a small team of data engineers/analysts.
- Review code, enforce best practices, and drive technical excellence.
- Build roadmap for data capabilities and automation.
Required Skills & Experience:
- 6 to 10 years of experience in market data, data engineering, or financial data operations.
- Strong understanding of:
1. Tick data
2. Corporate actions
3. Exchange market structures
4. Vendor feed behaviors
- Hands-on experience with:
1. Python (mandatory)
2. SQL and distributed databases
3. ETL frameworks
4. Message queues (Kafka/Pulsar a plus)
- Strong debugging skills for live and historical data issues.
Skills
Data Engineering, Python, SQL, Data Quality, Data Integrity, Data Storage, ETL, Data Management, Data Pipeline