Databricks Tech Lead/Architect
First seen by Alion on Sep 2, 2026.
Role Overview :
As a Databricks Tech Lead/Architect, you will serve as the technical cornerstone for our enterprise-grade data initiatives, driving the design and implementation of sophisticated data platforms.
You will operate at the intersection of business strategy and engineering, collaborating closely with cross-functional stakeholders, data engineers, and client leadership to translate complex requirements into scalable, high-performance architectures.
Your work will directly influence the organization's ability to derive actionable insights, ensuring that our data ecosystem remains robust, secure, and optimized for large-scale analytical processing. By championing best practices in cloud-native data engineering, you will play a pivotal role in accelerating digital transformation and delivering measurable business value across the enterprise.
Key Responsibilities :
- Architect and design end-to-end data solutions on Azure Databricks, ensuring seamless integration with existing cloud ecosystems to support high-volume data processing requirements.
- Lead the technical development of complex data pipelines using Apache Spark and Scala, optimizing performance and cost-efficiency for critical business workflows.
- Implement Delta Lake architectures to ensure data reliability, consistency, and governance across the entire data lifecycle for internal and external stakeholders.
- Mentor engineering teams by establishing coding standards, conducting rigorous code reviews, and fostering a culture of technical excellence and continuous improvement.
- Partner with business leaders to identify data-driven opportunities, providing expert guidance on cloud migration strategies and modern data stack adoption to drive competitive advantage.
Required Skillset :
- Demonstrated mastery in designing and deploying large-scale data architectures within cloud environments, specifically leveraging Azure Databricks to solve complex business problems.
- Advanced proficiency in Apache Spark and Scala, with a proven ability to write highly optimized, maintainable code for distributed computing environments.
- Exceptional ability to communicate intricate technical concepts to non-technical stakeholders, ensuring alignment between engineering efforts and overarching business objectives.
- Strong consultative mindset, capable of navigating ambiguous requirements and providing clear, actionable architectural roadmaps for diverse project teams.
- Proven track record of leading technical teams in a distributed or remote-first environment, maintaining high productivity and collaborative standards across multiple locations.
- A Bachelor's or Master's degree in Computer Science, Information Technology, or a related quantitative field, complemented by 12 to 20 years of progressive experience in data engineering and architecture.
Skills
Cloud Architect, Data Architect, Azure Databricks, Solution Architect, Scala, Distributed Computing