Data Engineer

Tennessee Oncology, one of the nation’s largest, community-based cancer care specialists, is home to one of the leading clinical trial networks in the country. Established 1976 in Nashville, Tennessee Oncology’s mission remains unchanged: To provide access to high-quality cancer care and the expertise of clinical research for all patients, at convenient locations within their community and close to their home. Our growing network of physicians and locations is based on this mission. Tennessee Oncology is committed to advancing both the science of detection and targeted treatments, and to making these advances available to every patient. We believe caring for cancer patients is a privilege.

Why Join Us? We are looking for talented and highly-motivated individuals who demonstrate a natural desire to support the meaningful work of community oncologists and the patients we serve.

Job Description:

The Data Engineer is a hands-on technical professional responsible for building, integrating, cleaning, and maintaining reliable data solutions that support reporting, analytics, and business intelligence operations. This position develops and supports data pipelines and data models in Databricks, works with SQL-based data sources and Parquet/Delta Lake storage, and delivers well-governed datasets for Power BI Report Server. The Data Engineer works closely with business stakeholders, analysts, application teams, and technology partners to translate data needs into practical, supportable solutions while maintaining strong standards for data quality, security, documentation and performance.

ESSENTIAL FUNCTIONS:

· Designs, builds, tests, deploys, and supports data ingestion and transformation pipelines in Databricks using SQL, Python, PySpark, notebooks, workflows, and related platform capabilities

· Connects to and integrates data from relational databases, APIs, files, cloud platforms, vendor systems, and other enterprise data sources

· Profiles, cleans, standardizes, enriches, and validates data so that downstream reporting and analytical datasets are accurate, consistent, and usable

· Develops and maintains scalable lakehouse data structures using Parquet and Delta Lake, including appropriate partitioning, schema management, table design, and file optimization

· Creates and maintains SQL queries, views, stored procedures, functions, and data models used for transformation, reconciliation, reporting, and analysis

· Builds curated datasets, semantic models, reports, dashboards, and visualizations for Power BI Report Server, working with report developers and business users to ensure accurate presentation, consistent metric definitions, reliable refreshes, and effective performance

· Implements data quality controls, reconciliation checks, exception handling, logging, and alerting throughout data pipelines

· Investigates pipeline failures, unexpected results, refresh problems, and source-system changes and coordinates corrective action through completion

· Monitors scheduled jobs, data freshness, storage utilization, and query performance and improves operational reliability, scalability, and cost efficiency

· Applies established data architecture, naming, metadata, lineage, retention, access, privacy, and lifecycle standards across data solutions

· Protects sensitive and regulated information by applying least-privilege access, secure data handling, appropriate masking, and organizational security and compliance requirements

· Works with business users and technical teams to define source behavior, business rules, reporting needs, and acceptance criteria

· Documents source-to-target mappings, business rules, transformations, dependencies, data models, operational procedures, and troubleshooting guidance

· Uses version control, code review, testing, deployment, and change-management practices to move data solutions safely between environments

· Identifies opportunities to standardize, automate, and simplify recurring data work and helps reduce manual data preparation and reporting processes

· Supports production data operations and participates in after-hours troubleshooting or scheduled implementation activities when needed

KNOWLEDGE, SKILLS & ABILITIES:

· Strong working knowledge of Databricks lakehouse concepts, notebooks, clusters or serverless compute, jobs/workflows, SQL Warehouses, and Delta Lake

· Strong SQL development skills, including complex joins, common table expressions, window functions, query optimization, data reconciliation, and relational data modeling

· Experience using Python and/or PySpark for data ingestion, transformation, validation, and automation

· Knowledge of Parquet file structure and behavior, including columnar storage, compression, schema evolution, partitioning, and small-file performance considerations

· Understanding of dimensional modeling, fact and dimension tables, star schemas, slowly changing dimensions, and semantic model design

· Working knowledge of Power BI Report Server, Power BI Desktop, Power Query, DAX, dataset refresh, gateways or connectivity, security, and report performance

· Understanding of ETL and ELT patterns, batch processing, incremental loading, change data capture, orchestration, dependency management, and restartable pipeline design

· Knowledge of data quality, governance, metadata, lineage, master/reference data, retention, privacy, and role-based access principles

· Ability to trace data from source through transformation to final report and explain discrepancies in clear, practical terms

· Strong troubleshooting, analytical, documentation, organization, and time-management skills

· Ability to manage competing priorities, communicate risks or blockers, and drive assigned work through completion

· Ability to explain technical data concepts and findings clearly to technical and non-technical audiences

· Self-motivated and directed, with the judgment to ask questions, validate assumptions, and escalate concerns when necessary

EDUCATION & EXPERIENCE:

· Associate’s or Bachelor’s degree in Information Technology, Computer Science, Data Engineering, Analytics, or a related field, or at least 4 years of equivalent experience in lieu of degree requirement.

· At least 3 years of experience in data engineering, business intelligence, data integration, database development, or a comparable role

· Demonstrated hands-on experience developing SQL-based data solutions and supporting production data pipelines

· Experience with Databricks, Delta Lake, Parquet, Python/PySpark, and Power BI strongly preferred

· Experience with Microsoft SQL Server and Power BI Report Server strongly preferred

· Experience working with healthcare data, regulated information, or complex clinical and operational source systems preferred

· Databricks, Microsoft Fabric, Azure Data Engineer, or Power BI certification is a plus

OOCUPATIONAL RISK CATEGORY:

PHYSICAL REQUIREMENTS:

· Work is primarily performed in an office or remote-office environment and requires extended use of a computer

· Must be willing and able to travel to clinic or business locations when necessary