Lead Data Engineer
First seen by Alion on Sep 17, 2026.
Key Responsibilities :
- Own the full data delivery lifecycle: procurement, vendor discussions, process setup, pipeline design and deployment (with logging and alerting), and database insertion - optimising throughout for minimal delivery latency.
- Build internal applications that simplify how quantitative developers query and consume data.
- Work across multiple datasets to deliver standardised, consistent data outputs.
- Break down large, ambiguous problems into scoped deliverables that hold up against deadlines.
- Translate business requirements into technical specifications, and clarify technical detail back to non-technical stakeholders.
- Separate genuine user needs from stated wants to prioritise effectively.
- Propose new ideas and improvements proactively, rather than waiting on direction.
- Step into senior-level responsibilities quickly when needed to support the team.
- Follow safe, disciplined data-handling practices throughout.
Skills & Experience :
Essential :
- Strong command of Python, Bash, and SQL in a Linux environment.
- Working understanding of low-level system design for efficient data delivery applications.
- Ability to explain technical concepts clearly to non-technical stakeholders.
- Hands-on experience with ClickHouse, including its Python, Bash, and C++ integrations.
- Familiarity with Jira and Agile ways of working.
- Proficiency with Git and standard version-control workflows.
- Comfortable developing over SSH.
- Solid technical grasp of market, alternative, and supporting datasets.
- Experience leading a team of 4 or more, delivering robust data solutions end-to-end.
Ideal :
- Experience handling 1+ TB of daily data processing.
- Track record building highly efficient, resource-optimised data pipelines.
- Current with hardware and software trends relevant to data engineering.
- Writes efficient, maintainable code without relying on LLM tooling.
- Prior experience in a fast-paced, high-availability proprietary trading environment.
Skills
Data Engineering, Python, Bash Scripting, SQL, Linux, Git, OpenSSH, Multithreading, C++, System Design