Lead Data Engineer - Databricks & Microsoft Fabric

Spyglass MTG · AI Consulting & Integration · Cloud Consulting & Migration · Python · ETL/ELT · 2 months ago

Smart Apply

Similar jobs

2,149,669open jobs

138,374companies

239,430added this week

Browse all

Salary

≈ $121k – $238k per year (Estimated)

Location

In office (Lincoln)

Seniority

Staff · 7+ years exp

Confirmed on the employer's own hiring board on Nov 10, 2026. First seen by Alion on Jul 23, 2026. Spyglass MTG scores B on the Alion truth index.

Overview

Company

Impact

Profile match

Spyglass MTG

Spyglass Solutions is a leading women-owned consulting firm specializing in Data, AI, and Security solutions on the Microsoft clouds. With over 15 years of experience, we offer expertise in AI Transformation, Data Platform Integration, Business Analytics, and more. Our services include M365 Migrations, Azure Platform solutions, Managed Services, and much more. As a Microsoft Solutions Partner with 5 Solutions Badges and FastTrack Ready status, we pride ourselves on delivering excellence to our clients. Our team of experts, including our experienced leadership team, is dedicated to your success. Whether you are looking to embark on an AI transformation journey, modernize on Azure, or enhance your security model, Spyglass Solutions is here to help. Contact us today to learn how we can pave the way for your business success in the digital age. Partner with us and experience the difference with Spyglass Solutions.

spyglassmtg.com · HQ: Lincoln, United States · Management Consulting · Est. 2018

Tech stack of Spyglass MTG

Description

Location: Remote (Preference for candidates located in New England)

Engagement: Contract or Contract-to-Hire

Lead Data Engineer

Databricks & Microsoft Fabric

Overview

Spyglass MTG is seeking a highly skilled Lead Data Engineer with strong expertise in Databricks and Microsoft Fabric to help design, build, and scale modern data platforms in Azure.

This role is both strategic and hands-on. The ideal candidate will lead the development of enterprise data solutions while actively building scalable pipelines, lakehouse architectures, and data platforms that support analytics, reporting, and AI initiatives.

The Lead Data Engineer will work closely with data architects, analysts, and business stakeholders to deliver high-performance data solutions that enable organizations to leverage their data more effectively.

Responsibilities

Design and implement modern data platforms using Databricks and Microsoft Fabric
Build and maintain scalable data pipelines and ingestion frameworks
Develop and optimize Spark and PySpark workloads within Databricks
Implement lakehouse and data warehouse architectures
Design and manage data models supporting analytics and reporting
Collaborate with data scientists and analytics teams to support AI and advanced analytics initiatives
Implement data quality, governance, and security best practices
Provide technical leadership and mentorship to data engineering team members
Optimize performance, reliability, and scalability of data platforms

Requirements

7+ years of experience in data engineering or data platform development
Strong hands-on experience with Databricks and Apache Spark
Experience implementing Microsoft Fabric data solutions
Proficiency in Python and PySpark
Experience working with Azure data services and cloud-based data platforms
Experience designing modern lakehouse or data warehouse architectures
Strong understanding of data pipeline design, ETL/ELT processes, and data modeling

Preferred

Experience supporting AI or machine learning workloads
Experience working in enterprise data environments
Familiarity with CI/CD and data platform automation

Free account

Stop reading job ads. Get the ones that fit.

One free account turns this page into a shortlist built around your stack, your level and your pay.

Match on every job. Stack, seniority, pay and location, scored against your profile.

2,149,669 open roles. Read straight off company career pages, refreshed every day.

Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.

3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.

Create a free account Continue with Google

Free forever. No card. Under a minute.

Your match

How well do you fit this role?

Two answers are enough for a real match. No account needed.

Check my fit

Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science

Similar stack

Same company

Lincoln

Data Scientist, Pricing 3 hours ago

Hybrid · Full-Time · 2+ years exp · Bachelor's Degree · London

Python

SQL

AI/ML

Machine Learning

Analytics

A/B Testing

Apply