Lead Data Engineer

We're a tech and product consultancy, building enterprise software products for blue-chip companies, multinationals and PLC's. The best thing about Red Badger is, of course, the team. We've been around for 15+ years now, and we are over 120 strong. We are really proud of our people; we support and learn a lot from each other, work really hard, and have fun doing it. We are a diverse group made up of 22 different nationalities, speaking 17 languages. Headquartered in London, we also have offices in Cape Town and Leeds.

The Role

We’re looking for a Senior Data Engineer to join our Data & AI delivery function. This is a hands-on, delivery-focused role: you'll work directly within a team of data engineers and a Tech Lead to design and build complex data solutions for our clients, from discovery through to production. Based in Leeds with a national remit, you'll pair regularly with other engineers and bring strong technical judgement to the problems you're solving.

The work we do for our clients involves the core complexities of their business, and the technology we use varies from project to project.

You love solving complex problems, and customer value is your north star. You're comfortable working across ambiguity, and you're a technology polyglot with an evangelist's enthusiasm for Agile ways of working.

Day to day, you'll be predominantly hands-on: building pipelines, designing data models, and pairing with other engineers on the team. You'll also contribute to technical decisions, support your Tech Lead in shaping solutions, and help to mentor and grow other engineers.

Key Responsibilities

Discipline Expertise

• Technology breadth: Production-grade experience across AWS and/or Azure native services, with strong hands-on Databricks and/or Snowflake, using the right tool for the job across the full data lifecycle.
• On AWS: comfortable across core storage, processing and data management services (e.g. S3, Glue, Redshift, Lake Formation)
• On Azure: comfortable across core storage, processing and data management services (e.g. Fabric, Synapse, Purview)
• Strong hands-on experience in Databricks and/or Snowflake across ingestion, transformation and serving
• Data capabilities: Confident across the full data lifecycle, OLTP and OLAP, batch and streaming pipelines, data quality, governance, metadata and cataloguing, through to analytics and decision-making.
• Architecture patterns: Understands medallion, data mesh/fabric, and when RAG or serverless patterns apply, working within an established reference architecture.
• Data Product mentality: You build with discoverability, addressability and security in mind, focused on outcomes rather than just moving data.
• AI: You understand how AI capability is built and deployed on modern data platforms (Databricks, Snowflake) and cloud-native services, and can implement AI-enabled features such as RAG pipelines with appropriate safeguards.
• Hands-on mindset: Fluent in SQL and Python/PySpark (or Scala/R). Comfortable with open formats including Parquet, Iceberg and JSON

Agile Ways of Working

• Contribute actively to agile ceremonies, helping to shape and improve the process
• Pair regularly with other engineers, including designers and cross-functional team members
• Advocate for lean development, building for what's required now
• Write tested, production-grade, maintainable code

Consulting with Clients

• Build strong working relationships with client technical stakeholders
• Communicate progress and technical trade-offs clearly to non-technical audiences when needed
• Support the upskilling of client teams in modern data practices where relevant

Growth & Collaboration

• Support and mentor less experienced engineers through pairing and code review
• Bring the Red Badger culture with you: low-ego, collaborative, and open to challenge
• Contribute technical opinions confidently, while staying open to being wrong

Your experience 

It’d be great if:

• You are a Quality Advocate: You take pride in the integrity of your work and champion quality assurance and performance tuning.
• You are an Industry Steward: A lifelong learner who keeps a pulse on industry standards and best practices.
• You care deeply about high-quality work that delivers real value
• You're comfortable with ambiguity and solving complex problems collaboratively
• You bring strong expertise in your craft, alongside a willingness to keep learning
• You communicate clearly and build trust fast with engineering peers and client stakeholders
• You're pragmatic, adaptable, outcome-focused and low-ego in multidisciplinary teams

It’d be amazing if you can also: 

• Show a track record of delivering complex data projects end-to-end
• Articulate technical trade-offs in terms of business impact, right tool for the job
• Demonstrate experience across more than one of the core platforms (AWS, Azure, Databricks, Snowflake)
• Be part of the public face of the company at meetups, conferences, etc.

Diversity and Inclusion

Red Badger is committed to creating a diverse and inclusive environment, and is proud to be an equal opportunity employer. We hire, employ, train, promote and compensate regardless of gender, gender identity, race, disability, sexual orientation, age, creed and colour. We are fully focused on equality, and we believe the fascinating characteristics that make us all different enable us to continue doing amazing work for our clients.