Sr Data Engineer - Databricks

Georgia-Pacific is seeking a Senior Databricks Data Engineer to join its Data Platform team. The role focuses on building and enhancing the Databricks platform, delivering reliable and scalable data pipelines, establishing reusable engineering patterns, and supporting governed data assets for analytics, advanced analytics, and AI use cases.

Responsibilities

• Design, build, deploy, and support Databricks-native data engineering solutions, including Delta Lake pipelines, Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader ingestion pipelines, Unity Catalog resources, and CI/CD automation
• Develop reliable, governed data platforms and pipelines that improve the accuracy, consistency, accessibility, and trustworthiness of enterprise data assets
• Create automated tools, reusable templates, and self-service patterns that accelerate data development, ingestion, feature engineering, and consumption
• Operate and improve the platform through performance monitoring, incident resolution, migrations, upgrades, observability, and cost optimization aligned with long-term scalability and resilience goals
• Partner with product owners, operations teams, architects, and delivery teams to identify common use cases, prioritize platform enhancements, and expand adoption
• Integrate Databricks pipelines with enterprise governance, metadata, lineage, classification, certification, ownership, and data-quality capabilities
• Develop and curate governed data assets for AI and advanced analytics use cases, including semantic models, APIs, Genie Spaces, and AI agents
• Implement validation rules, monitoring, and remediation processes that improve the fitness and reliability of critical data assets

Skills

• Experience designing, building, and supporting production-grade Databricks, Delta Lake, and Apache Spark pipelines for batch and streaming workloads
• Experience developing Databricks Jobs (i.e. Lakeflow Declarative Pipelines, Structured Streaming) and multi-task workflows, including scheduling, parameters, dependencies, retries, notifications, and operational monitoring
• Experience implementing Databricks infrastructure and environment-aware deployment automation using Terraform, Databricks Asset Bundles, or other Infrastructure as Code practices, Git, and CI/CD workflows
• Experience implementing data governance and access controls with Unity Catalog, including catalogs, schemas, tables, views, external locations, storage credentials, lineage, and fine-grained permissions
• Experience using Python and Spark diagnostic tools, including job logs, Spark UI, query plans, partitioning, join optimization, and Adaptive Query Execution, to troubleshoot and optimize production data workloads
• Must have legal authorization to work permanently in the United States for any employer without requiring a visa transfer or visa sponsorship
• Experience building production ingestion pipelines with Databricks Auto Loader
• Advanced Python experience applied to data engineering, automation, reusable libraries, and platform capabilities
• Experience with open table formats such as Delta Lake, Apache Hudi, or Apache Iceberg
• Experience developing governed data assets for AI, semantic modeling, APIs, advanced analytics, Genie Spaces, or AI agents
• Experience with AWS, Microsoft Azure, or Google Cloud and native storage services such as Amazon S3, Azure Data Lake Storage Gen2, OneLake, or Google Cloud Storage

Benefits

• Hybrid work arrangement with an expectation of three (3) days in office per week
• Medical benefits
• Dental benefits
• Vision benefits
• Flexible spending accounts
• Health savings accounts
• Life insurance
• AD&D insurance
• Disability benefits
• Retirement benefits
• Paid vacation/time off
• Educational assistance
• May also include infertility assistance
• May also include paid parental leave
• May also include adoption assistance

Company Overview