Senior Software Engineer

Responsibilities:

• Design and implement high-throughput, event-driven architectures to power near-real-time data pipelines for ingesting into Azure + Databricks Lakehouse
• Build cloud-native data pipelines to support batch ingest workloads.
• Configure Unity Catalogue by designing Bronze/Silver/Gold medallion architecture and optimise for centralised data governance, fine-grained access control, and end-to-end lineage tracking across Databricks workspaces.
• Drive engineering excellence by setting standardised DevOps practices and optimising data engineering code for performance.
• Identify and design solutions to potential data processes that can be improved with automation.
• Partner with domain teams to onboard new capability use cases, mentor engineers, and contribute to reusable blueprints and reference implementations.

Requirements:

• 7+ years of software engineering experience focused on data infrastructure and backend systems.
• Hands-on experience with the Microsoft Azure data platform: Azure Data Factory, Event Hubs, Blob Storage, ADLS Gen2 and Azure DevOps for orchestration and CI/CD.
• Production experience with Azure Databricks and Delta Lake.
• Production experience with PySpark for distributed data processing and transformation.
• Streaming frameworks (Kafka or comparable) for real-time data paths, including CDC and incremental batch patterns.
• Strong communication skills with the ability to influence senior stakeholders and lead technical decisions across global teams.

Must Have Skills:

• Data Platform Architecture (Lakehouse, Delta Lake, Azure Databricks).
• Kafka / Streaming Frameworks.
• Strong SQL and Python Proficiency.
• Experience with cloud-native tools.

Nice-to-Have:

• Building data infrastructure for AI use cases.
• Prior experience in large-enterprise data integration or platform engineering environments.
• Prior experience in enterprise platform migrations.

Skills

azure, databricks, delta lake, kafka, pyspark, python