{"id":2141301,"url":"https://alion.io/job/mastek-lead-data-engineer","title":"Lead Data engineer","company":{"id":59435,"name":"Mastek","domain":"mastek.com","url":"https://alion.io/company/mastek","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SuccessFactors","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":96000,"max_usd":171000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":28},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"Databricks","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"SQL","optional":false},{"name":"Amazon CloudWatch","optional":true},{"name":"Amazon S3","optional":true},{"name":"AWS Glue","optional":true},{"name":"AWS Lambda","optional":true},{"name":"AWS Step Functions","optional":true},{"name":"CI/CD","optional":true},{"name":"Delta Lake","optional":true},{"name":"Git","optional":true},{"name":"GitLab CI","optional":true},{"name":"IAM","optional":true},{"name":"Jenkins","optional":true},{"name":"Linux","optional":true},{"name":"Snowflake","optional":true},{"name":"Spark","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-10-06T00:00:00Z","employer_posted_date":"2026-10-06","last_verified_at":"2026-10-12T19:29:12Z","board_verified":true,"closed_at":null,"days_open":7,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":7},"description":"Data Engineer\nSupport and Maintenance\nDatabricks | AWS | Python | PySpark | SQL\nLocation: Leeds, UK /Reading, UK\nWork Pattern: Hybrid\nTravel Requirements: Mostly Remote - Travel to customer and Mastek offices as required\nSecurity Clearance: Not Mandatory - Good to Have\nImmediate Joiners\nAbout Mastek\nMastek Limited is a global provider of enterprise AI, digital, and cloud services, enabling clients to achieve measurable and sustainable returns on their technology investments. With a presence in over 40 countries and a workforce of nearly 5,000 professionals, Mastek delivers innovative technology solutions that help organisations accelerate transformation and unlock business value.\nThrough our Lead with AI approach, we integrate intelligence across our solutions and operations, enabling ethical, scalable, and domain-driven AI adoption. We partner with industry leaders including Oracle, Salesforce, Microsoft, AWS, Snowflake, and Databricks, delivering outcomes across sectors such as Healthcare, Financial Services, Retail, Manufacturing, Higher Education, and the Public Sector.Guided by our core values of Trust, Value, and Velocity, we foster a culture of collaboration, innovation, continuous learning, and career growth.\nAbout the role\nWe are looking for a hands-on Data Engineer to run and maintain production data pipelines on Databricks and AWS. You will keep scheduled processing reliable, resolve incidents, deliver permanent fixes and improve the performance of existing data services.\nThe role combines L2 and L3 production support with data engineering. You will work with service management, platform teams, data owners and downstream users to keep data accurate and available within agreed service levels and delivery deadlines.\nKey responsibilities\nMonitor scheduled jobs, ingestion processes and downstream loads. Identify failures, delays and unusual data volumes, assess the business impact and prioritise action against delivery deadlines.\nOwn incidents from investigation through recovery and closure. Diagnose issues in Python, PySpark, SQL, Databricks jobs and AWS integrations, and provide clear progress updates and timely escalation.\nRecover failed or incomplete processing through controlled retries, reruns and backfills. Check dependencies and reconcile outputs to prevent duplicate records, missing data or inconsistent downstream results.\nInvestigate data quality issues, schema changes and late or missing source files. Trace problems through pipeline stages and agree corrective action with source owners and data consumers.\nMaintain and improve existing Python, PySpark and SQL code. Deliver defect fixes and small enhancements, using code review and appropriate tests to protect existing processing rules.\nTune Spark jobs and SQL queries, including joins, partitioning, data skew and memory usage. Review Databricks compute usage and S3 file layouts to reduce processing time and unnecessary cost.\nTroubleshoot S3 access, IAM permissions, encrypted data access and connectivity with platform and security teams. Support runtime and dependency upgrades, controlled releases, rollback plans and post-release validation.\nAutomate health checks, reconciliation and routine support tasks. Improve alerts and track job failures, data freshness, processing duration and recurring incidents to guide preventive maintenance.\nMaintain runbooks, dependency maps, root cause analyses and support handovers. Check that new or changed pipelines have monitoring, support ownership and recovery procedures before accepting them into service.\nEssential skills and experience\nProven experience supporting and maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis and permanent remediation.\nStrong Python, PySpark and SQL skills, with the ability to understand unfamiliar code, diagnose defects and make maintainable changes.\nPractical Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI and performance troubleshooting.\nExperience with Spark SQL, Delta Lake, Parquet and Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files.\nHands-on AWS experience, particularly S3, IAM and CloudWatch, with an understanding of role-based access, KMS encryption and diagnosing data access failures.\nUnderstanding of batch and incremental processing, job dependencies, restartability, late-arriving data and safe reprocessing without duplicate results.\nExperience implementing data quality checks and source-to-target reconciliation, and investigating missing, duplicate or incorrect records in large datasets.\nWorking knowledge of Git, code reviews, CI/CD and controlled production deployments, including testing, rollback and release validation.\nExperience working within incident, problem and change management processes. Able to prioritise by business impact, maintain clear support records and communicate with technical and non-technical colleagues.\nDesirable experience\nAWS Glue, Lambda, Step Functions or other orchestration services; Linux and shell scripting; Terraform; and delivery tools such as GitLab CI or Jenkins.\nMonitoring and alerting automation, capacity planning, cloud cost optimisation and disaster recovery exercises for data services.\nSupporting healthcare, public sector or other regulated data environments, including access controls, audit trails and secure handling of sensitive information.\nTransitioning pipelines from project delivery into live support, with practical knowledge transfer and documented operational acceptance.\nWorking expectations\nTake ownership of issues through to resolution, work calmly during service disruption and raise risks early. Follow agreed access and change controls, and coordinate maintenance and recovery with the teams affected. Any on-call or out-of-hours support arrangements will be agreed for the role.\nWhy Join Mastek?\nWork on large-scale digital transformation programmes across public and private sector clients.\nCollaborate with highly skilled and diverse teams across multiple disciplines.\nGain exposure to modern technologies, frameworks, and delivery methodologies.\nAccess ongoing learning, professional development, and certification opportunities.\nContribute to meaningful projects that have real-world impact.\nBe part of an inclusive culture that values innovation, teamwork, and continuous improvement.\nEnjoy clear career progression opportunities within a global technology organisation.\nOur UK Benefits Include\nCompetitive salary and annual bonus opportunities\nHybrid and flexible working options\nGenerous annual leave entitlement plus bank holidays\nCompany pension scheme with employer contributions\nPrivate medical insurance (for self and dependants)\nLife assurance cover\nEmployee Assistance Programme (EAP)\nWellbeing initiatives and gym support\nLearning and development programmes, certifications, and professional training\nEmployee recognition and reward schemes\nCycle to Work scheme\nEmployee referral bonus programme\nEnhanced family-friendly policies\nDiversity & Inclusion\nAt Mastek, we are committed to building a diverse workforce and creating an inclusive environment where everyone can succeed. We welcome applications from all backgrounds and are proud to be an equal opportunities employer.\nWe believe diversity drives innovation and are dedicated to fostering a workplace where every employee feels valued, respected, and empowered to make an impact.\nApply today and help shape the future with Mastek.","description_format":"text","description_chars":7591,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Annual leave","Continuous learning","Flexible schedule","Health insurance","Professional development"],"hiring_locations":[{"name":"United Kingdom","iso":"GB","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Enterprise Apps Implementation Partners"],"lifecycle":[{"event":"open","at":"2026-10-09T03:15:10Z"}],"visa":[{"country":"GB","licensed_sponsor":true,"evidence":"Licensed UK visa sponsor (Skilled Worker, Senior or Specialist Worker, Graduate Trainee)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Skilled Worker; Global Business Mobility: Senior or Specialist Worker; Global Business Mobility: Graduate Trainee","cap_exempt":false,"checked_at":"2026-10-10T06:21:00+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":6,"expected_fill_days":29,"reasons":["conf:12","velocity","win:early"],"computed_at":"2026-10-12T06:55:09Z"},"pay":null,"html_url":"https://alion.io/job/mastek-lead-data-engineer","json_url":"https://alion.io/job/mastek-lead-data-engineer.json","meta":{"generated_at":"2026-10-13T02:15:21Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3307,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-14T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":59435},"rest":"https://alion.io/mcp/rest/get_company?id=59435"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmastek-lead-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmastek-lead-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmastek-lead-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/mastek-lead-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmastek-lead-data-engineer"}]}