Data Governance & Metadata Engineer

Key Responsibilities

Metadata Engineering & Automation

• Model and document data structures, metadata relationships, and end-to-end lineage across data platforms (including lakehouse architectures).
• Configure and maintain metadata workflows, connectors, and governance assets.
• Build automations and integrations in Java and Python (strong pandas expertise required).
• Integrate technical metadata from Databricks (Spark, Delta Lake, Unity Catalog) and Microsoft Purview into governance tools.
• Operate and troubleshoot metadata services on Linux (logs, services, deployments).

Data Governance

• Maintain data domains, dictionaries, glossaries, classification models, and stewardship structures.
• Define and enforce metadata standards and data quality rules across analytical platforms.
• Support impact analysis, remediation, and governance alignment for data products and lakehouse use cases.

Collibra / Purview

• Maintain metadata, lineage, stewardship models, and workflows in Collibra.
• Integrate Collibra with technical metadata sources (e.g. Databricks, Unity Catalog, Microsoft Purview, SQL engines) via APIs, scanners, or pipelines.
• Align governance models between Collibra and Purview (glossaries, classifications, lineage where applicable).
• Provide onboarding, training, and high-quality documentation.

Cross-Functional Work

• Collaborate with data engineers, platform teams, and product owners to ensure consistent governance standards in Databricks, Unity Catalog, and Purview.
• Assess risks, impacts, and compliance aspects in data-related projects.
• Translate technical platform concepts (Spark, lakehouse, catalogs, semantic layers) into clear governance artefacts.

Required Skills

• Hands-on experience with metadata platforms and governance tooling.
• Strong understanding of data modelling, metadata architectures, lineage, and catalog concepts.
• Proficiency in Java and Python (mandatory pandas expertise).
• Solid Linux skills.
• Experience with REST APIs, SQL, and metadata extraction.
• Practical experience with Databricks and Unity Catalog.
• Familiarity with Microsoft Purview concepts (scanning, classifications, lineage).
• Knowledge of data governance and data quality frameworks.
• Strong documentation and communication skills.
• Experience with Git and versioning workflows.

Nice to Have

• Advanced hands-on experience with Databricks (Spark, Delta Lake, Unity Catalog, jobs).
• Experience integrating Purview and Collibra in hybrid governance setups.
• Azure services and orchestration tools experience.
• Metadata scanning, MDM, or lineage tooling experience.
• Collibra workflow development.
• Understanding of data security, classification, and access control models.
• Familiarity with industry lineage standards and open metadata approaches.