Senior Data Engineer (Databricks)

Join us on the R&D Softwareteam as a Data Engineer (Databricks), and help shape the future of safer, more efficient, and more reliable operations across the globe. Start your journey with Anova today!

Where you’llwork: This is a hybrid role based out of our Porto office. In practice, most of your work can be done remotely, with occasional in-office time in Porto for team collaboration - a flexibility our engineers consistently tell us theyvalue.

Job Duties and Responsibilities:

You will build and run the Databricks pipelines that turn real-time telemetry and platform data into reliable, well-governed data assets - the master data that reporting, analytics and machine learning across Anova all depend on.

Collaborate for success

• Deliver Databricks ETL projects end to end, from requirements through topipelines running in production.

• Translate business goals into data solutions and help stakeholders make the right choices about data.

• Contribute to technical decisions, take a significant share of the implementation, and monitorthe pipelines you own once they are live.

Build the One Anova data stream

• Work with real-time telemetry from industrial IoT sensors deployed across the globe.

• Build the BI aggregations that bring data from across platforms together into consistent, reusable data assets.

• Your pipelines are the backbone for our internal natural-language digital assets that letsany employee query Anova's data without writing SQL. The reliability, freshnessand clarity of what you publish directly determineswhether that experience can be trusted.

• Publish and maintaindata assets as master data for the organization.

Engineer with AI assistance

• Use agentic coding tools - Claude Code, Copilot, Cursor and similar - as a normal part of daily delivery.

• Hold AI-generated code to the same bar as any other code. You are accountable for what you ship.

• Keep repositories, testsand documentation structured soboth people and agents can work in them effectively.

Advocate for quality

• Contribute to and continuously adapt best practices and Ways of Working around data engineering, testingand pipeline operations.

• Maintain clear data lineage and definitions for the assets you own - as AI agents increasingly query this data directly, untraceableor ambiguous data becomes a governance risk, not just a data-quality one.

• Treat data quality as a feature: tests, expectationsand monitoring, so problems surface before stakeholders findthem.

Minimum Requirements -

• Bachelor's degree in Computer Science, Data Engineering, Data Science, or a related quantitative field orequivalent combination of education and experience

• 5+ years of experience in data engineering or a closely related role, with hands-on production experience in Databricks (6-8 years preferred).

• Significant experiencebuilding data workloads in Databricks, with a very goodunderstanding of PySparkand Delta Lake.

• Strong SQL - window functions, complex joins and query tuning are everyday tools for you.

• Experience with streaming or incremental ingestion (Structured Streaming, Auto Loader, or equivalent) and the patterns that keep it correct: idempotency, checkpointing and schema evolution.

• Data modelling for BI and analytics.

• Good understanding of testing and CI/CD for Databricks workflows, alongside the software engineering and DevOps basics - git, code review, linters, unit tests and CI/CD pipelines are things you use daily.

• Data quality practice: testing data as well as code, using pipeline expectations, dbttests or similar.

• Comfortable using agentic coding tools, with a clear view of where they help and where they need supervision.

• Proficient in written and spoken English.

Preferred Qualifications -

• Databricks platform depth beyond the basics: Lakeflowpipelines (formerly Delta Live Tables), LakeflowJobs, Unity Catalog for governance and lineage, and infrastructure as code with Declarative Automation Bundles or Terraform.

• Performance and cost optimization on Databricks: cluster sizing, Photon, liquid clustering, and partitioning.

• The wider Azure data ecosystem: Event Hubs or Data Factory.

• Master data management or data governance practice: clear ownership, stewardshipand agreed definitions for shared data assets.

• Domain experience in industrial, energyor IoT settings.