Senior Data Engineer, 6-month contract

Job Description:

• Work in Agile teams to deliver the committed backlog against acceptance criteria
• Design, build, and operate ingestion and orchestration pipelines in Azure Data Factory
• Develop ETL/ELT in Azure Databricks using a medallion architecture for analytics, reporting, and AI
• Lead code reviews and set engineering standards
• Participate in estimating, planning, testing, deployments, and documentation
• Shape data architecture decisions, including metric definitions, semantic models, and data serving AI agents
• Apply data governance, security, and access controls according to client classification and residency requirements
• Use AI-assisted development tools responsibly to accelerate delivery without compromising quality or security
• Communicate progress, open items, and issues to stakeholders, including management
• Stay current with relevant technology developments
• Deliver robust data platforms and AI-enabled solutions on Microsoft Azure supporting decision-making, autonomous execution, and integrated operations
• Mentor team members and lead by doing

Requirements:

• 7+ years’ experience in data engineering or software development
• 3+ years hands-on with Azure Databricks or Spark
• Experience actively mentoring junior team members
• Expert-level Python and PySpark, and SQL
• Experience with Azure Data Factory, Azure Databricks, Delta Lake, Workflows, Unity Catalog, Azure Data Lake Storage, and Azure SQL
• Experience with dimensional and lakehouse data modelling
• Experience integrating data via APIs, OData, JDBC/ODBC, and files
• Experience with Azure AI services, including Azure AI Foundry and Azure OpenAI, and preparing data for AI agents
• Experience with secure CI/CD and infrastructure-as-code using tools such as Terraform, Bicep, or Databricks Asset Bundles
• Experience across DEV, UAT, and PROD using Azure DevOps or GitHub, with sound Git branching practices
• Experience with AI-assisted development and reviewing generated code
• Knowledge of data quality, testing, security, and privacy in regulated or restricted-data environments
• Experience designing end-to-end data solutions and recommending frameworks and technologies
• Experience with Agile and Kanban delivery in mixed-discipline teams
• Self-starter, organized, and able to work with minimal supervision
• Bachelor’s degree in computer science, Engineering, or a related field, or equivalent education and experience
• Nice to have: experience integrating enterprise systems
• Nice to have: data management and governance concepts, including data quality, cataloguing, lineage, master data, and stewardship

Benefits:

• Fully remote position
• 6-month contract