{"id":3529068,"url":"https://alion.io/job/storeforce-senior-data-platform-engineer","title":"Senior Data Platform Engineer","company":{"id":4420602,"name":"StoreForce","domain":"storeforce.com","url":"https://alion.io/company/storeforce","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"BambooHR","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Toronto, Canada"],"countries":["CA"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":68000,"max_usd":121000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":90},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Anthropic","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Claude","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"LLM","optional":false},{"name":"OpenAI","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Azure SQL Database","optional":true},{"name":"LLM Guardrails","optional":true},{"name":"Microsoft Entra ID","optional":true},{"name":"PostgreSQL","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-10-30T15:22:41Z","employer_posted_date":"2026-10-30","last_verified_at":"2026-11-01T13:14:32Z","board_verified":true,"closed_at":null,"days_open":2,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":2},"description":"OVERVIEW\nReporting to the VP, Technical Operations, the Senior Data Platform Engineer is a senior, hands-on individual contributor on our Engineering, Data Platform team. This is a full-time, remote role open to candidates based in the United States or Canada, with no direct reports.\nStoreForce helps specialty retailers run their stores, from workforce scheduling to performance analytics, across tens of thousands of locations worldwide. We are building a new data platform on Azure Databricks that turns the operational data behind those stores into governed metrics, embedded analytics, and AI-driven insight for our customers.\nAs a Senior Data Platform Engineer, you will build, run, and scale that platform: Spark and Delta Lake pipelines from our multi-tenant SQL Server estate, Unity Catalog governance with a shared semantic layer, and the analytics agent on top. You will work in a small platform team, alongside an implementation partner during the initial build, and own the platform’s growth from the first customers to the full base.\nWHAT YOU'LL DO\n1. Lakehouse Engineering & Data Pipelines\nBuild and run the lakehouse on Azure Databricks, including change tracking and change data capture (CDC) ingestion from SQL Server through Lakeflow Connect.\nDevelop Spark and Delta Lake transformations in Lakeflow Spark Declarative Pipelines and jobs, along with orchestration and serving layers.\nImplement and maintain the medallion and dimensional data model in Delta Lake.\nTune Spark jobs, Photon, liquid clustering, and storage for performance and cost.\nMeasure page and report latency against Lakebase and Databricks SQL, adding a columnar serving store only if the numbers require it.\n2. Semantic Layer & Data Governance\nMaintain the governed semantic layer in Unity Catalog (Metric Views) so that dashboards, embedded analytics, applications, and AI agents consume the same certified metrics.\nEnforce per-tenant row and column security at query time.\nImplement StoreForce’s retail KPI logic in Metric Views, including fiscal calendars, 53-week comparisons, comp days, and targets.\nReconcile Metric View results to the existing application’s numbers.\n3. Platform Scale & Tenant Onboarding\nScale the platform by configuration: onboard tenants and add KPIs by repeating the established pattern.\nMap each tenant’s configurable KPI columns to one governed schema.\nExtend and fix the pattern when it needs a new code path.\n4. Data Quality, Reliability & Cost\nEnforce data quality in the pipeline: expectations, data contracts, reconciliation to source, and freshness and volume checks.\nMaintain lineage and Lakehouse Monitoring, treating data incidents like production incidents.\nKeep the platform reliable in production, favoring serverless and managed services over undifferentiated infrastructure.\nDeliver everything as code through CI/CD, with close attention to compute sizing and DBU spend.\n5. AI-Powered Analytics\nBuild and operate the LLM-powered analytics agent that answers natural-language questions against the semantic layer.\nUse Databricks AI tooling alongside Anthropic Claude and OpenAI models.\nEnsure tenant isolation, runtime guardrails, and an automated evaluation suite that acts as a quality gate.\nUse AI-assisted development to ship quickly and effectively.\n6. Collaboration & Platform Ownership\nWork alongside the implementation partner during the initial build and take ownership of the platform at handover.\nServe as the go-to engineer for data at StoreForce, contributing design decisions backed by evidence.\nKeep engineering standards and runbooks current.\nSupport the teams consuming the platform and triage data incidents.\nOther duties as assigned by your manager and senior leadership.\nQUALIFICATIONS\nCompetencies Required:\nTechnical Depth- Hands-on expert across the Databricks and Spark core who writes production code every day\nOwnership- Takes end-to-end accountability for the platform, from build through production support\nData Quality Mindset- Treats data correctness and data incidents with the rigor of production software\nEvidence-Based Decisions- Measures before deciding and backs design choices with data\nLearning Agility- Picks up new tools quickly and embraces AI-assisted development\nCollaboration & Communication- Works effectively with partners and consuming teams; explains trade-offs clearly to technical and non-technical audiences\nEXPERIENCE / KNOWLEDGE\n5+ years building data pipelines or backend systems, including at least 2 years running a modern data stack in production\nHands-on production experience with Databricks: Unity Catalog, Delta Lake, Lakeflow Jobs and Spark Declarative Pipelines (or their predecessors, Workflows and Delta Live Tables), Databricks SQL, and deploying through Databricks Asset Bundles or an equivalent as-code practice\nStrong Apache Spark skills: PySpark and Spark SQL, Structured Streaming, reading query plans, diagnosing shuffle and skew, and tuning jobs and compute for performance and cost\nStrong SQL (window functions, complex joins, query optimization) and strong Python for data work\nSolid data modeling across medallion, dimensional, and incremental patterns, with experience tuning Delta tables (liquid clustering, change data feed, OPTIMIZE and VACUUM strategy)\nExperience implementing data quality and observability (expectations, contracts, reconciliation, lineage, monitoring, alerting)\nHands-on experience with modern LLM providers (Anthropic Claude and OpenAI), building AI features against their APIs\nFluent in modern software engineering practices (version control, automated testing, CI/CD) and able to explain everyday trade-offs such as serverless vs. classic compute or streaming vs. batch; infrastructure or DevOps specialization is not required.\nPREFERRED QUALIFICATIONS\nDatabricks Certified Data Engineer Associate or Professional\nExperience deploying LLMs against structured data in production on Databricks (Genie, the agent framework, Model Serving for external models, MLflow evaluation), particularly evaluation, guardrails, and CI gating\nExperience with multi-tenant SaaS data on Databricks, including per-tenant isolation with Unity Catalog row filters and column masks, and catalog design for many tenants\nExperience ingesting from SQL Server and Azure SQL Managed Instance with Change Tracking or CDC over private networking\nExperience serving curated data to always-on applications from Lakebase or PostgreSQL\nExperience delivering embedded analytics inside a product, not only internal business intelligence\nExperience taking over a platform from a consultancy or another team and running it well afterward.\nTECHNICAL ENVIRONMENT\nIngestion- Lakeflow Connect with SQL Server change tracking\nStorage- Delta Lake on Azure Data Lake Storage Gen2 in StoreForce’s own tenancy, in open formats so the data outlives any engine\nTransformation- PySpark, Spark SQL, and Lakeflow Spark Declarative Pipelines\nGovernance- Unity Catalog for governance, lineage, and the semantic layer (Metric Views)\nOrchestration & Compute- Lakeflow Jobs, with serverless compute where it fits\nServing- Databricks SQL for BI and embedded analytics; Lakebase or PostgreSQL as the always-on serving store\nInfrastructure & Security- Everything as code with Databricks Asset Bundles and Terraform; private networking with Entra ID throughout\nYou don’t need experience with every technology: bring hands-on depth across the Databricks and Spark core and be ready to learn the rest. This is a data engineering role, not infrastructure.\nKEY PERFORMANCE INDICATORS (KPIs)\nPipeline success rates, data freshness against agreed SLAs, and availability of serving layers\nReconciliation of governed KPIs to the existing application’s numbers, data quality check pass rates, and number and severity of data incidents\nTime to detect and resolve data incidents, with root causes addressed and runbooks updated\nDashboard, report, and embedded analytics latency against agreed targets\nTime to onboard a new tenant and to add a new KPI using the standard pattern\nCompute and DBU spend per tenant, and cost trends against budget\nEvaluation suite pass rates, answer accuracy, guardrail effectiveness, and zero cross-tenant data exposure\nSuccessful handover from the implementation partner, current standards and runbooks, and satisfaction of teams consuming the platform.\nWHY WORK WITH US AT STOREFORCE\nStoreForce is growing quickly - it’s the kind of place you can make a real impact. We thrive on change, collaboration, and trust; And are fueled by a mutual passion for Specialty Retail. At StoreForce you’ll have the chance not only to work with some amazing people, but also with some of the world’s best retail brands.\nWe're remote and offer flexibility in your schedule;\nEmployer-paid health benefits including dental, prescription drug coverage, and a Health Spending Account;\nRRSP Matching with First Time Home Buyer support*\nCompany paid training and professional development;\nExtended Long Weekends and Religious Observance Days\nEmployee Referral Program\nSpotlight Recognition Bonus Program\nDedicated Employee Assistance Program (EAP)\n*Speak with HR for full program details.\nStoreForce is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind. We are committed to the principle of equal employment opportunity for all individuals and to providing individuals with a work environment that is diverse and inclusive. This starts by welcoming and encouraging applications from all individuals including those with disabilities. Accommodations are available on request for candidates taking part in all aspects of the selection process.\nAt StoreForce, we leverage AI tools to support a fair and efficient recruitment process. This may include AI-assisted resume screening to help identify top candidates and AI note taking during interviews. All hiring decisions are made by real people. AI is used only as a supplemental tool to enhance, not replace, human judgment.\nWe never accept or respond to speculative applications from third party recruiters and work only with preferred partners.","description_format":"text","description_chars":10104,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Professional development"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Professional Services","Human Resources","HR & Payroll Software"],"lifecycle":[{"event":"open","at":"2026-10-30T15:22:41Z"}],"visa":[],"liveness":{"score":95,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.946,"p_room":1,"age_days":1,"expected_fill_days":28,"reasons":["conf:1","urgency","win:early"],"computed_at":"2026-11-01T06:10:25Z"},"pay":null,"html_url":"https://alion.io/job/storeforce-senior-data-platform-engineer","json_url":"https://alion.io/job/storeforce-senior-data-platform-engineer.json","meta":{"generated_at":"2026-11-01T16:01:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":24654,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-02T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4420602},"rest":"https://alion.io/mcp/rest/get_company?id=4420602"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fstoreforce-senior-data-platform-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fstoreforce-senior-data-platform-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fstoreforce-senior-data-platform-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/storeforce-senior-data-platform-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fstoreforce-senior-data-platform-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3529068}},{"tool":"search_jobs","arguments":{"company":"storeforce.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3529068},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}