{"id":720993,"url":"https://alion.io/job/shyftlabs-data-engineer-4","title":"Data Engineer","company":{"id":685982,"name":"Shyftlabs","domain":"shyftlabs.io","url":"https://alion.io/company/shyftlabs","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"A","score":91,"open_postings":25,"ghost_share":0,"stale_share":0.36,"repost_share":0,"time_to_fill_p50_days":39,"computed_at":"2026-11-01T06:10:25Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Calgary, Canada"],"countries":["CA"],"hiring_countries":["CA"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":69000,"max_usd":133000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":17},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Databricks","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"Docker","optional":false},{"name":"ElasticSearch","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"Google BigQuery","optional":false},{"name":"Hadoop","optional":false},{"name":"Looker","optional":false},{"name":"Python","optional":false},{"name":"Scrum","optional":false},{"name":"SQL","optional":false},{"name":"AI Agents","optional":true},{"name":"Apache Kafka","optional":true},{"name":"Druid","optional":true},{"name":"Spark","optional":true}],"status":"live","first_seen_at":"2026-03-13T19:29:11Z","employer_posted_date":"2026-03-13","last_verified_at":"2026-11-01T14:17:51Z","board_verified":true,"closed_at":null,"days_open":232,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":232},"description":"Position Overview\nWe are looking for an experienced and versatile Data Engineer to join our dynamic and fast-growing team. If you are passionate about data, solving complex problems, and working directly with enterprise stakeholders to translate business needs into scalable technical solutions, this role could be the perfect fit.\nShyftLabs is a growing data product company that was founded in early 2020 and works primarily with Fortune 500 companies. We deliver digital solutions built to help accelerate the growth of businesses across various industries by focusing on creating value through innovation.\nIn addition to strong technical expertise, we are seeking someone with strong business awareness and the ability to lead client and stakeholder communication. The ideal candidate will be comfortable collaborating with enterprise-level clients, translating complex technical concepts into business outcomes, and ensuring alignment between engineering execution and strategic objectives.\nJob Responsibilities\nDesign, build, and maintain scalable and reliable batch and real-time ETL/ELT data pipelines using cloud services such as GCP Dataflow, Cloud Functions, Pub/Sub, and Cloud Composer.\n\nArchitect and implement robust data infrastructure capable of handling high-volume data ingestion and processing.\n\nDevelop and manage our central data warehouse in Google BigQuery.\n\nDesign and implement data models, schemas, and table structures optimized for performance, scalability, and long-term maintainability.\n\nWrite clean, efficient, and maintainable SQL and Python code to transform raw data into curated, analysis-ready datasets.\n\nBuild reliable transformation workflows that support analytics, reporting, and data science initiatives.\n\nMonitor, troubleshoot, and optimize data infrastructure to ensure high performance, reliability, and cost efficiency.\n\nImplement BigQuery best practices, including partitioning, clustering, query optimization, and materialized views.\n\nBuild and maintain curated data models that serve as the “source of truth” for business intelligence and reporting.\n\nEnsure data is optimized and readily accessible for BI tools such as Looker and other analytics platforms.\n\nImplement automated data quality checks, validation rules, and monitoring frameworks to ensure the integrity and reliability of data pipelines and warehouse systems.\n\nEstablish processes for data governance, observability, and lineage tracking.\n\nWork closely with software engineers, data analysts, and data scientists to understand their data requirements and provide the necessary infrastructure and data products.\n\nLead and support client and stakeholder communication, working with enterprise clients to translate business needs into scalable data solutions.\n\nPartner with product teams and leadership to ensure that technical data solutions align with business strategy and client expectations.\n\nTake ownership of data platforms and architecture decisions, helping shape the future direction of our analytics and data infrastructure.\n\nIdentify opportunities to improve data reliability, automate workflows, and generate new insights through data.\n\nContribute to a collaborative, high-performing engineering culture with strong communication and teamwork.\n\nBasic Qualifications\n5+ years of hands-on experience in data engineering, data integration, or data platform development.\n\nDegree in Computer Science, Engineering, Mathematics, or related STEM discipline.\n\nStrong programming and query skills in SQL and Python.\n\nExperience working with distributed version control systems such as Git in an Agile/Scrum environment.\n\nExperience designing and orchestrating ETL pipelines, particularly with Databricks.\n\nExperience working within cloud environments (GCP, AWS, or Azure).\n\nExperience with database systems such as MongoDB and Elasticsearch.\n\nStrong understanding of data warehousing and dimensional modeling methodologies.\n\nHands-on experience with Airflow and Hadoop.\n\nExperience using Docker for containerized workflows and reproducible environments.\n\nAbility to identify opportunities to improve data quality, reliability, and automation.\n\nStrong business awareness and communication skills, with the ability to collaborate with both technical teams and business stakeholders.\n\nExperience within the retail industry is a plus.\n\nPreferred Qualifications\nMaster’s degree in Computer Science, Engineering, or related discipline.\n\nExperience working with enterprise-scale data platforms and Fortune 500 clients.\n\nFamiliarity with Druid and its Python API, including Kafka integrations.\n\nStrong experience using Apache Spark for large-scale data processing.\n\nExperience designing real-time streaming data architectures.\n\nExperience working with AI-driven platforms, data infrastructure supporting AI/ML systems, or agentic AI workflows","description_format":"text","description_chars":4847,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Consulting & Integration"],"lifecycle":[{"event":"open","at":"2026-09-11T07:11:31Z"}],"visa":[],"liveness":{"score":12,"band":"cold","label":"Long shot","p_open":1,"p_active":0.434,"p_room":0.28,"age_days":232,"expected_fill_days":39,"reasons":["conf:0","velocity","win:tail","crowd:"],"computed_at":"2026-11-01T06:10:25Z"},"pay":null,"html_url":"https://alion.io/job/shyftlabs-data-engineer-4","json_url":"https://alion.io/job/shyftlabs-data-engineer-4.json","meta":{"generated_at":"2026-11-01T14:27:51Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":22165,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-02T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":685982},"rest":"https://alion.io/mcp/rest/get_company?id=685982"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fshyftlabs-data-engineer-4"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fshyftlabs-data-engineer-4"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fshyftlabs-data-engineer-4"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/shyftlabs-data-engineer-4\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fshyftlabs-data-engineer-4"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":720993}},{"tool":"search_jobs","arguments":{"company":"shyftlabs.io","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":720993},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}