Lead Data Engineer - Databricks & Microsoft Fabric

Spyglass MTG • Python • ETL/ELT • 16 days ago
Apply now
Similar jobs
746,022open jobs
44,798companies
108,231added this week
Browse all
Salary
≈ $123k – $241k per year (Estimated)
Location
In office (Lincoln)
Seniority
Staff · 7+ years exp

Confirmed on the employer's own hiring board on Sep 24, 2026. First seen by Alion on Jul 23, 2026.

Overview
Company
Impact
Profile match

Spyglass MTG

Spyglass Solutions is a leading women-owned consulting firm specializing in Data, AI, and Security solutions on the Microsoft clouds. With over 15 years of experience, we offer expertise in AI Transformation, Data Platform Integration, Business Analytics, and more. Our services include M365 Migrations, Azure Platform solutions, Managed Services, and much more. As a Microsoft Solutions Partner with 5 Solutions Badges and FastTrack Ready status, we pride ourselves on delivering excellence to our clients. Our team of experts, including our experienced leadership team, is dedicated to your success. Whether you are looking to embark on an AI transformation journey, modernize on Azure, or enhance your security model, Spyglass Solutions is here to help. Contact us today to learn how we can pave the way for your business success in the digital age. Partner with us and experience the difference with Spyglass Solutions.
spyglassmtg.com • HQ: Lincoln, United States • Cybersecurity • Information Technology • Data & Analytics • Est. 2018

Description

Location: Remote (Preference for candidates located in New England)

Engagement: Contract or Contract-to-Hire

Lead Data Engineer

Databricks & Microsoft Fabric

Overview

Spyglass MTG is seeking a highly skilled Lead Data Engineer with strong expertise in Databricks and Microsoft Fabric to help design, build, and scale modern data platforms in Azure.

This role is both strategic and hands-on. The ideal candidate will lead the development of enterprise data solutions while actively building scalable pipelines, lakehouse architectures, and data platforms that support analytics, reporting, and AI initiatives.

The Lead Data Engineer will work closely with data architects, analysts, and business stakeholders to deliver high-performance data solutions that enable organizations to leverage their data more effectively.

Responsibilities

  • Design and implement modern data platforms using Databricks and Microsoft Fabric
  • Build and maintain scalable data pipelines and ingestion frameworks
  • Develop and optimize Spark and PySpark workloads within Databricks
  • Implement lakehouse and data warehouse architectures
  • Design and manage data models supporting analytics and reporting
  • Collaborate with data scientists and analytics teams to support AI and advanced analytics initiatives
  • Implement data quality, governance, and security best practices
  • Provide technical leadership and mentorship to data engineering team members
  • Optimize performance, reliability, and scalability of data platforms

Requirements

  • 7+ years of experience in data engineering or data platform development
  • Strong hands-on experience with Databricks and Apache Spark
  • Experience implementing Microsoft Fabric data solutions
  • Proficiency in Python and PySpark
  • Experience working with Azure data services and cloud-based data platforms
  • Experience designing modern lakehouse or data warehouse architectures
  • Strong understanding of data pipeline design, ETL/ELT processes, and data modeling

Preferred

  • Experience supporting AI or machine learning workloads
  • Experience working in enterprise data environments
  • Familiarity with CI/CD and data platform automation

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
746,022 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
Lincoln
≈ $97k – $207k per year (Estimated) • In office • Full-Time • San Antonio
Python
SQL
Databases
PostgreSQL
PostGIS
AI/ML
MLFlow
Amazon SageMaker
DevOps
CI/CD
AWS
Amazon S3
Cybersecurity
SOC 2
Least Privilege
SpaceTech
GDAL
Apply
$160k – $200k per year • In office • Full-Time • 2+ years exp • San Francisco
SQL
Databases
Google BigQuery
BigQuery
Analytics
Tableau
ETL/ELT
A/B Testing
Looker
Management
Discord
Apply
$137k – $213k per year • In office • Full-Time • 3+ years exp • Bachelor's Degree • Foster City
Python
AI/ML
Copilot
Spark
ChatGPT
Scikit-learn
TensorFlow
PyTorch
NeuralProphet
Time Series Forecasting
Recommender Systems
Machine Learning
Apply
≈ $120k – $207k per year (Estimated) • Equity • Remote (United States) • Full-Time • 5+ years exp • Bachelor's Degree • United States
Python
SQL
AI/ML
Airflow
Machine Learning
DevOps
Terraform
GCP
Azure
CI/CD
AWS
Docker
Kubernetes
IAM
Cybersecurity
Crowdstrike
Analytics
Superset
Apply
≈ $116k – $223k per year (Estimated) • Hybrid • Full-Time • 7+ years exp • PhD • United States
Python
SQL
Databases
Databricks
Apache Kafka
Microsoft Fabric
AI/ML
Spark
AI Agents
Machine Learning
DevOps
Azure
CI/CD
Analytics
ETL/ELT
Data Vault
Dimensional Modeling
Master Data Management
Management
Agile
Apply
≈ $21k – $53k per year (Estimated) • In office • Full-Time • Chennai • Gurgaon • Mumbai
Python
SQL
Python
pySpark
Databases
Databricks
Delta Lake
Azure SQL Database
AI/ML
Spark
DevOps
Azure
Analytics
Power BI
ETL/ELT
Talend
Azure Data Factory
Management
Agile
Apply
≈ $16k – $36k per year (Estimated) • Remote (EAEU) • Moscow
Python
1C
Databases
PostgreSQL
Redis
ClickHouse
KeyDB
RabbitMQ
MinIO
Apache Kafka
AI/ML
Machine Learning
DevOps
Terraform
Ansible
Helm
Cilium
Loki
Traefik
Fluentd
K3s
Prometheus
Velero
VictoriaMetrics
CI/CD
GitOps
ArgoCD
Git
Docker
Kubernetes
Nginx
Grafana
Gitea
AWX
Helmfile
Vector
GitLab
Amazon S3
IAM
DNS
Cybersecurity
Keycloak
Calico
LDAP
SIEM
Management
ITSM
Naumen
Apply
Remote (Greece) • Part-Time • Athens
Python
JavaScript
SQL
Python
Django
Databases
PostgreSQL
DevOps
GCP
DigitalOcean
Heroku
Azure
AWS
Apply
Auditor 1 day ago
In office • Full-Time • Athens
Python
SQL
AI/ML
Machine Learning
Analytics
Power BI
ETL/ELT
Apply
≈ $87k – $226k per year (Estimated) • In office • Internship • Bachelor's Degree • Petah Tikva • Yokneam
Python
Verilog
C++
SystemVerilog
Perl
AI/ML
Copilot
ChatGPT
DevOps
Linux
Chips/EDA
UVM
Apply
≈ $110k – $223k per year (Estimated) • Remote (United States) • Contractor
Python
Python
pySpark
Databases
Microsoft Fabric
AI/ML
Copilot
Spark
Chips/EDA
Synopsys SpyGlass
Analytics
ETL/ELT
Data Vault
Apply
In office • Contractor
SAS
Databases
Amazon Redshift
AI/ML
Hadoop
Amazon SageMaker
DevOps
AWS
Amazon S3
Analytics
ETL/ELT
Collibra
Management
Agile
Apply
≈ $96k – $179k per year (Estimated) • Hybrid • 5+ years exp • Lincoln
DevOps
Azure
Chips/EDA
Synopsys SpyGlass
Apply
Enterprise Architect 16 days ago
≈ $148k – $277k per year (Estimated) • Hybrid • Lincoln
Databases
Databricks
Microsoft Fabric
DevOps
Azure
Apply
≈ $96k – $179k per year (Estimated) • Hybrid • 5+ years exp • Lincoln
DevOps
Azure
Chips/EDA
Synopsys SpyGlass
Apply
Enterprise Architect 16 days ago
≈ $148k – $277k per year (Estimated) • Hybrid • Lincoln
Databases
Databricks
Microsoft Fabric
DevOps
Azure
Apply
Controls Systems Tech 4 years ago
≈ $75k – $145k per year (Estimated) • In office • Full-Time • 5+ years exp • Associate's Degree • Lincoln
Apply
See all jobs
This is one of many
746,022 more open roles from verified company boards, updated every day.
Browse all jobs Get matched roles