{"id":2924075,"url":"https://alion.io/job/scrumconnect-data-test-engineer-sc-cleared","title":"Data Test Engineer (SC Cleared)","company":{"id":3861039,"name":"Scrumconnect Consulting","domain":"scrumconnect.com","url":"https://alion.io/company/scrumconnect","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Zoho Recruit","truth_index":{"grade":"B","score":75,"open_postings":24,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-22T06:10:10Z"}},"role":"Data Science","role_family":"Data Science","seniority":null,"employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Newcastle upon Tyne, United Kingdom"],"countries":["GB"],"hiring_countries":["IN","GB"],"hiring_countries_total":2,"salary":null,"salary_estimate":{"min_usd":66000,"max_usd":151000,"period":"year","method":"role_country_seniority_unknown","sample_n":183},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon EC2","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Glue","optional":false},{"name":"CI/CD","optional":false},{"name":"Docker","optional":false},{"name":"DynamoDB","optional":false},{"name":"GitLab","optional":false},{"name":"GitLab CI","optional":false},{"name":"IAM","optional":false},{"name":"pySpark","optional":false},{"name":"Terraform","optional":false},{"name":"Java","optional":true},{"name":"Python","optional":true},{"name":"Spark","optional":true},{"name":"SQL","optional":true}],"status":"live","first_seen_at":"2026-07-17T00:00:00Z","employer_posted_date":"2026-07-17","last_verified_at":"2026-10-22T21:45:27Z","board_verified":true,"closed_at":null,"days_open":97,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":97},"description":"About Scrumconnect\nScrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure, and scalable technology solutions for public and private sector clients. This is a fully remote role based in India, supporting UK operations and client engagements. The successful candidate will work closely with UK leadership teams and must be comfortable operating within UK regulatory, governance, and delivery frameworks.\nAbout the Role\nWe are looking for an experiencedTest Manager to lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services.\nThis role combineshands-on technical leadership withtest strategy ownership. You will be responsible for defining and driving the overall testing approach across automated pipelines, data workflows, cloud infrastructure, and release delivery while managing risks, improving testing standards, and mentoring engineering teams.\nThe ideal candidate will have strong experience inBig Data and Data Lake environments, including testing both structured and unstructured datasets, and will be comfortable operating in highly automated CI/CD-driven delivery environments.\nYou will work closely with engineering, DevOps, architecture, and stakeholder teams to ensure quality is embedded throughout the software delivery lifecycle.\nKey Responsibilities\nTest Strategy & Leadership\nDefine, implement, and manage the overall test strategy across cloud-native data platforms and applications\n\nLead end-to-end testing activities across data pipelines, cloud services, APIs, and distributed systems\n\nEstablish quality standards, governance processes, and testing best practices across teams\n\nDrive a quality-first engineering culture across the programme\n\nProvide leadership, mentoring, and coaching to Test Engineers and QA team members\n\nAutomated Testing & Frameworks\nDesign, implement, and oversee scalable automated test frameworks for data and application testing\n\nBuild and maintain automated test suites across AWS cloud infrastructure and data workflows\n\nEnhance existing automation frameworks to improve reliability, coverage, maintainability, and reporting\n\nEnsure automated testing is fully integrated within GitLab CI/CD pipelines for rapid feedback and release confidence\n\nBig Data & Data Lake Testing\nLead testing activities across Big Data and Data Lake solutions containing structured and unstructured datasets\n\nValidate large-scale data ingestion, transformation, and processing workflows\n\nEnsure data integrity, transformation accuracy, reconciliation, and performance under load\n\nOversee PySpark-based data validation and interrogation activities on AWS EMR\n\nSupport testing and monitoring of workflows orchestrated through Apache Airflow\n\nDrive root cause analysis for production data issues and platform defects\n\nRisk Management & Reporting\nIdentify, assess, and communicate quality risks across releases and environments\n\nAnalyse test execution metrics, defects, trends, and quality indicators\n\nTranslate technical findings into clear, prioritised reporting for technical and non-technical stakeholders\n\nProvide regular status reporting to programme leadership and client stakeholders\n\nCI/CD & DevOps\nCollaborate with DevOps and engineering teams to improve release quality and deployment confidence\n\nEnsure automated test execution within GitLab CI/CD pipelines\n\nSupport infrastructure testing and validation within Terraform-managed cloud environments\n\nContribute to continuous improvement initiatives across testing and delivery processes\n\nTechnology Stack\nApache Airflow, Amazon Athena, AWS S3, AWS Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark / PySpark, Terraform, Docker\n\nKey Skills\nStrong proficiency in Python, SQL, Java, and scripting languages such as Bash\n\nHands-on experience testing Big Data platforms and Data Lake solutions\n\nStrong experience with AWS EMR, Apache Airflow, and Jupyter Notebooks\n\nExperience performing PySpark-based data interrogation, validation, and testing\n\nStrong knowledge of AWS services including S3, Glue, Athena, EMR, CloudWatch, IAM, DynamoDB, EC2, and ECR\n\nExperience designing, implementing, and managing enterprise-scale automated test frameworks\n\nStrong understanding of CI/CD practices using GitLab pipelines\n\nExperience with infrastructure-as-code tools such as Terraform\n\nFamiliarity with Docker and containerised testing environments\n\nUnderstanding of dimensional data models and slowly changing dimensions\n\nExperience validating structured and unstructured data workflows\n\nStrong analytical and root cause analysis skills\n\nAbility to communicate technical risks clearly to stakeholders at all levels\n\nExperience leading and mentoring QA/Test Engineering teams\n\nSkills & Experience Required\nStrong background in automated testing for cloud-native and data-centric platforms\nExperience testing large-scale data pipelines, transformation processes, and distributed systems\nExperience managing or leading QA/testing teams in Agile delivery environments\nStrong stakeholder management and communication skills\nExperience establishing test governance, reporting, and quality metrics\nStrong problem-solving and decision-making capability\nExperience working within regulated or security-conscious delivery environments\n SFIA Level 4\nActive SC clearance (mandatory)\nDiversity and Inclusion\nAt Scrumconnect Consulting, we believe diversity drives innovation and stronger outcomes for our teams and clients. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds, experiences, and perspectives.","description_format":"text","description_chars":5920,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"},{"name":"United Kingdom","iso":"GB","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Professional Services","Travel & Tourism","E-learning"],"lifecycle":[{"event":"open","at":"2026-10-20T13:00:11Z"}],"visa":[{"country":"GB","licensed_sponsor":true,"evidence":"Licensed UK visa sponsor (Skilled Worker)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Skilled Worker","cap_exempt":false,"checked_at":"2026-10-22T06:27:10+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":8,"band":"cold","label":"Long shot","p_open":1,"p_active":0.282,"p_room":0.28,"age_days":97,"expected_fill_days":36,"reasons":["conf:2","stale_co","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-22T06:10:10Z"},"pay":null,"html_url":"https://alion.io/job/scrumconnect-data-test-engineer-sc-cleared","json_url":"https://alion.io/job/scrumconnect-data-test-engineer-sc-cleared.json","meta":{"generated_at":"2026-10-22T22:45:53Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":8148,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3861039},"rest":"https://alion.io/mcp/rest/get_company?id=3861039"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fscrumconnect-data-test-engineer-sc-cleared"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fscrumconnect-data-test-engineer-sc-cleared"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fscrumconnect-data-test-engineer-sc-cleared"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/scrumconnect-data-test-engineer-sc-cleared\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fscrumconnect-data-test-engineer-sc-cleared"}]}