{"id":3086829,"url":"https://alion.io/job/randstad-sr-data-engineer","title":"Sr Data Engineer","company":{"id":19351,"name":"Randstad Canada","domain":"randstad.ca","url":"https://alion.io/company/randstad","size_band":null,"is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":["Mississauga, Canada"],"countries":["CA"],"hiring_countries":["CA"],"hiring_countries_total":1,"salary":{"min":66,"max":74,"currency":"CAD","period":"hour","gross":null,"usd_annual":106000},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Git","optional":false},{"name":"MS SQL","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Azure","optional":true},{"name":"Azure Data Factory","optional":true},{"name":"dbt","optional":true},{"name":"Great Expectations","optional":true},{"name":"HIPAA","optional":true},{"name":"HL7","optional":true}],"status":"live","first_seen_at":"2026-10-15T20:12:24Z","employer_posted_date":null,"last_verified_at":"2026-10-15T20:12:24Z","board_verified":false,"closed_at":null,"days_open":6,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":6},"description":"Our client, is seeking an experienced, detail-oriented Sr Data Engineer to join their platform modernization team.\n\nIn this technical, hands-on role, you will build pipelines, transformations, and data validation tools to support the modernization of a high-stakes healthcare pharmacy compliance platform using Databricks. Working within an Agile delivery pod under a technical lead, you will engineer high-volume ingestion workloads across a multi-tier Delta Lake architecture (Bronze, Silver, Gold) while porting and re-implementing existing SQL Server stored procedure logic into PySpark and Spark SQL. Because this platform drives financial and regulatory decisions, engineering efforts prioritize data accuracy, automated testing, parity reconciliation against legacy outputs, and full traceability across every pipeline step.\n\nDuration: 6-Month Contract (100% Remote)\n\nWork Arrangement: 100% Remote\n\nAdvantages\n100% Remote Flexibility: Enjoy complete remote work flexibility while contributing to a mission-critical platform modernization initiative.\n\nModern Lakehouse Architecture: Hands-on technical execution utilizing Databricks, PySpark, Spark SQL, and Delta Lake design patterns.\n\nHigh-Impact Regulatory Domain: Drive data pipelines for a platform handling critical healthcare and pharmacy compliance workflows where precision and data quality are paramount.\n\nFocused 6-Month Engagement: High-intensity consulting assignment centered on core legacy-to-cloud platform migration and reconciliation.\n\nResponsibilities\nDelta Lake Pipeline Engineering: Build and maintain high-volume batch and intraday ingestion pipelines into a Delta Lake architecture (Bronze, Silver, Gold layers).\n\nStored Procedure Migration: Port, translate, and re-implement legacy SQL Server stored procedure business logic into optimized PySpark and Spark SQL code.\n\nData Quality & Reconciliation: Develop automated test suites, data quality checks, and output-parity reconciliation jobs comparing new lakehouse results against legacy system outputs.\n\nPerformance Tuning & Optimization: Optimize Spark job execution by tuning partition strategies, file sizing, join types, skew handling, and cluster compute parameters.\n\nIdempotency & Late Data Handling: Construct idempotent, re-runnable data pipelines designed to gracefully manage late-arriving or corrected data updates.\n\nCI/CD & Engineering Standards: Maintain version control in Git, participate in peer code reviews, build CI/CD pipeline automation, and document technical design specs.\n\nQualifications\nData Engineering Depth: 6+ years of dedicated professional experience in data engineering, including 3+ years of hands-on production experience with Databricks and Apache Spark.\n\nPython & PySpark Proficiency: Strong skills independently writing, testing, and tuning production-grade Python and PySpark scripts.\n\nSQL & T-SQL Expertise: Advanced proficiency in SQL and T-SQL, with the demonstrated ability to interpret, dissect, and translate complex SQL Server stored procedures.\n\nDelta Lake Expertise: Practical experience with Delta Lake capabilities, including MERGE/upsert operations, incremental loads, schema evolution, time travel, and table maintenance.\n\nSpark Execution Knowledge: Deep understanding of the Spark execution model (jobs, stages, shuffles, join strategies, memory allocation) and performance troubleshooting.\n\nDevOps & Testing: Proficiency using Git, automated data testing frameworks, and CI/CD tools for data pipeline delivery.\n\nCommunication Skills: Strong written communication skills with a focus on clear technical documentation and team collaboration.\n\nPreferred Assets & Nice-to-Haves\nExperience with healthcare, pharmacy, payer, or life-sciences data (e.g., HL7, ePrescribe, NCPDP, claims, NDC drug data, or HIPAA-regulated PHI).\n\nDirect experience with SQL Server to Databricks / Lakehouse migrations.\n\nFamiliarity with Azure Data Factory, Azure Storage, Delta Live Tables, Databricks Workflows, and Unity Catalog.\n\nExposure to data quality tools such as Great Expectations or transformation frameworks like dbt.\n\nActive Databricks Certified Data Engineer (Associate or Professional) designation.\n\nSummary\nIf you are a tech-savvy Sr Data Engineer who pairs 6+ years of enterprise experience with 3+ years of production Databricks/PySpark mastery, T-SQL stored procedure translation skills, and a commitment to data accuracy and testing, this 100% remote 6-month contract opportunity is an ideal match. Bring your Lakehouse engineering precision, Spark tuning skills, and migration expertise to our team today!\n\nRandstad Canada is committed to fostering a workforce reflective of all peoples of Canada. As a result, we are committed to developing and implementing strategies to increase the equity, diversity and inclusion within the workplace by examining our internal policies, practices, and systems throughout the entire lifecycle of our workforce, including its recruitment, retention and advancement for all employees. In addition to our deep commitment to respecting human rights, we are dedicated to positive actions to affect change to ensure everyone has full participation in the workforce free from any barriers, systemic or otherwise, especially equity-seeking groups who are usually underrepresented in Canada's workforce, including those who identify as women or non-binary/gender non-conforming; Indigenous or Aboriginal Peoples; persons with disabilities (visible or invisible) and; members of visible minorities, racialized groups and the LGBTQ2+ community.\n\nRandstad Canada is committed to creating and maintaining an inclusive and accessible workplace for all its candidates and employees by supporting their accessibility and accommodation needs throughout the employment lifecycle. We ask that all job applications please identify any accommodation requirements by sending an email to  to ensure their ability to fully participate in the interview process.\n\nThis posting is for existing and upcoming vacancies.","description_format":"text","description_chars":6030,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Biotechnology"],"lifecycle":[{"event":"open","at":"2026-10-22T13:43:23Z"}],"visa":[],"liveness":{"score":52,"band":"ok","label":"Likely open","p_open":1,"p_active":0.516,"p_room":1,"age_days":6,"expected_fill_days":29,"reasons":["seen:6","agency","win:early","comp:brand"],"computed_at":"2026-10-22T16:23:43Z"},"pay":{"stated_usd_annual":106000,"is_top_pay":false},"html_url":"https://alion.io/job/randstad-sr-data-engineer","json_url":"https://alion.io/job/randstad-sr-data-engineer.json","meta":{"generated_at":"2026-10-22T16:23:43Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16650,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":19351},"rest":"https://alion.io/mcp/rest/get_company?id=19351"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Frandstad-sr-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Frandstad-sr-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Frandstad-sr-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/randstad-sr-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Frandstad-sr-data-engineer"}]}