{"id":2637333,"url":"https://alion.io/job/sitetracker-senior-data-scientist-agentic-ux-design","title":"Senior Data Scientist (Agentic UX), Design","company":{"id":7358,"name":"Sitetracker","domain":"sitetracker.com","url":"https://alion.io/company/sitetracker","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"B","score":81,"open_postings":4,"ghost_share":0,"stale_share":0.25,"repost_share":0.5,"time_to_fill_p50_days":78,"computed_at":"2026-10-15T05:46:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":{"min":145000,"max":215000,"currency":"USD","period":"year","gross":null,"usd_annual":215000},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"A/B Testing","optional":false},{"name":"AI Agents","optional":false},{"name":"Claude Code","optional":false},{"name":"Cursor","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Scrum","optional":false},{"name":"Slack","optional":false}],"status":"live","first_seen_at":"2026-10-15T14:56:40Z","employer_posted_date":"2026-10-15","last_verified_at":"2026-10-16T00:51:54Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"The Opportunity\nAs a Senior Data Scientist on the Design team, you will play a pivotal role in transforming how some of the world's most critical infrastructure companies operate. The experiences you shape for both humans and agents will directly influence how complex projects are planned, deployed, and managed at scale.\nWe are rethinking what excellent UX means in an agentic world. When the interface is a model, the quality of the experience is no longer carried primarily by pixels and flows - it is carried by behavior. What the agent retrieves, what it decides, what it says, when it refuses, and how reliably it does all of that across thousands of real situations. Those are measurement problems, experimentation problems, and evaluation problems. They are your problems.\nThis role is ideal for someone who combines statistical rigor with a strong builder mentality - someone energized by ambiguity, fluent in model behavior, and obsessed with what \"good\" actually means and how to prove it. You will set the standard for how agent experiences are designed, instrumented, and held to a quality bar at Sitetracker - hands-on, technically deep, and relentlessly connected to customer outcomes.\nYou will be embedded in a scrum team - designing, measuring, and delivering alongside Engineering and Design on a day-to-day basis. You will own day-to-day decisions for your experience area, mentor others on rigor and method, and ship to production. For someone who wants to build, measure, and elevate the craft around them, this is the role.\nWhat You'll Do\nOwn Your Domain and Deliver High-Impact Solutions\nYou will take ownership of day-to-day decisions for your experience area. You will take priorities from leadership and define solutions independently - making informed tradeoff decisions across user needs, agent behavior, technical constraints, and business goals. You will define the leading indicators that tell you whether an experience is working, scope work, prioritize effectively, and own the consequences of your decisions. You will ship fast without sacrificing quality, knowing when to ship on signal and when you need more.\nStudy the Market and the Technology\nYou will stay close to where product and AI capabilities are heading, not just what Sitetracker can do today. You will maintain running awareness of the competitive landscape across multiple verticals, identify gaps and opportunities, and produce analysis that connects directly to product and behavior decisions. You will develop a deep understanding of the critical infrastructure industries Sitetracker serves - telecommunications, EV charging, renewables, and utilities - and the workflows professionals in those industries rely on daily, because you cannot evaluate an agent's answers without understanding what a right answer looks like in the field. You will speak the technical language - fluent enough in model behavior, retrieval, platform capabilities, and enterprise architecture to hold your own as a peer in deep technical discussions, not a passenger.\nInvestigate Complex Problems\nYou will investigate complex, cross-team problems, synthesizing signal from logs, traces, evals, and customer behavior to separate root causes from symptoms. You will own investigations end to end - instrumenting what isn't measured yet, designing the analysis, and moving from finding to action quickly. You will work directly with customers - including those with complex needs or strong opinions - running pilots, mining how they actually use the product, and turning behavioral patterns into design decisions. You will show up to customer calls thoroughly prepared with account context and the data to back it.\nDesign the Full Agentic Experience\nYou will design comprehensive solutions that think in systems and behavior, not screens. You will define what the agent should do across the full distribution of inputs it will see - not just the happy path - and design the context architecture, decision paths, and communication patterns that get it there. You will map where the agent surfaces across the product, define the behavioral contract for each surface, and account for how changes ripple across the experience. Sitetracker is a Salesforce ISV - our web experience is built on SLDS-based components, while our mobile experience is purpose-built outside of SLDS. You will design for the full range of our users across both - from field technicians on mobile to program managers running complex dashboards - and for the agent that now sits between them and the work.\nBuild and Ship\nThe discipline is changing. The most effective people on a product team are no longer handing off static artifacts and waiting for someone else to bring their ideas to life. They are shipping code, building AI graphs, measuring the quality of their solutions, and delivering outcomes directly. Sitetracker is at the front edge of this evolution, and this role will help define what it looks like.\nYou are a builder. A notebook is one tool in the kit, not the destination. You will ship code to production using AI-assisted development tools like Claude Code, Cursor, and Scout itself, build and refine the data pipelines and eval harnesses that keep quality honest, and verify implementations directly. You will build flexible prototypes and instrumentation, loop engineers in for review, and write code that works within the existing codebase. You will be a trusted partner to your engineering counterparts, actively engaging in engineering Slack channels and contributing to technical discussions.\nArchitect and Evaluate Scout AI Experiences\nYou will architect Scout graphs and design sophisticated AI interactions. You will be fluent in the technical language - able to reason about graph logic, context retrieval, grounding, and AI behavior independently. You will design Scout's decision paths, context architecture, and communication patterns, and define how Scout surfaces throughout the broader product. Most of all, you will own quality: you will write comprehensive evals across multiple dimensions - correctness, grounding, safety, refusal, latency, tone - define what success means quantitatively, and track it over time in Watch. You will design the offline and online evaluation strategy for your area, build the datasets that power it, and stay current with the broader AI landscape so your eval design keeps pace with what the models can and can't do.\nValidate and Measure Quality\nThis is the center of the role. You will strategically choose when and how to validate, and you will bring real method to it. You will design experiments - A/B tests, holdouts, shadow evaluations - to de-risk before customers are involved and to learn from real behavior once they are. You will distinguish a real effect from noise, size experiments correctly, and resist shipping on anecdote. You will run usability and behavior reviews for complex agent features across verticals, use AI-assisted workflows to build high-fidelity prototypes that test real Scout behavior, and run customer betas and previews - including with customers who have more complex needs or stronger opinions. You will establish metrics connected to commercial outcomes and consider your work done when it's succeeding, not when it ships.\nDrive Adoption and Outcomes\nYour work is not done when a feature ships. It's done when it succeeds. You will track whether what you built is adopted and whether it drives the commercial outcomes it was designed to achieve. You will build the dashboards and analysis that make adoption and outcomes legible to the rest of the org, and you will apply what you learn to iterate, adjust, and improve.\nMentor and Elevate the Team\nYou will raise the rigor bar around you. You will review others' work - evals, experiments, analyses, and agent designs - and provide substantive, actionable feedback that improves output. You will teach the team how to define quality, design an eval, and read an experiment honestly. You will model the \"build in public\" culture by sharing work early and often across broad channels, inviting critique, and showing how work evolves based on input. You may manage a dedicated intern, setting clear expectations and ensuring they do meaningful work.\nThe Skills You'll Have\nAct with Agency\nThrive in fast-moving environments and deliver high-impact outcomes at pace\nIndependently identify ambiguous challenges and take ownership of driving them to resolution\nChampion new ideas, influence stakeholders, and rally support around strategic initiatives\nIdentify workflows where AI tools and emerging technologies accelerate delivery, and make building the default way you work\nMeasurement, Experimentation, and Evaluation\nDefine what \"good\" means for an agentic experience and turn it into metrics, evals, and datasets that hold up under scrutiny\nDesign and analyze experiments - A/B tests, holdouts, shadow runs - with correct sizing, clean reads, and honest conclusions\nBuild offline and online evaluation systems for LLM behavior across correctness, grounding, safety, tone, and latency\nApply statistical and ML fluency to real, messy production behavior - distinguishing signal from noise and root cause from symptom\nAgentic UX and Behavior Design\nTurn complex enterprise workflows into agent experiences people can actually use without training\nDesign context architecture, decision paths, and communication patterns, and reason about how they shape behavior across the full input distribution\nWork directly with customers - running pilots, mining usage, and iterating based on how they actually behave, including customers with complex needs or in delicate situations\nMake tradeoff decisions across user needs, agent behavior, technical constraints, and business goals, and connect those decisions to commercial outcomes\nTechnical Fluency and Building\nOpen PRs that ship work to production, build functional prototypes and pipelines, and use AI-assisted development tools as a primary working medium\nBuild instrumentation and eval harnesses that are reusable and work within the existing codebase\nSpeak the technical language as a peer - comfortable in discussions about platform architecture, retrieval, grounding, and engineering tradeoffs\nArchitect Scout graphs and reason about graph logic, context retrieval, and AI behavior independently\nCompetitive & Market Awareness\nMaintain running awareness of the competitive landscape across multiple verticals\nIdentify gaps and opportunities and produce analysis that connects to product and behavior decisions\nTrack market trends, adjacent products, and emerging tools, connecting what's happening in the market to what Sitetracker should be doing\nRegularly contribute to competition and industry Slack channels with substantive observations\nProduct & Industry Knowledge\nDeep product knowledge across your area and adjacent areas\nUnderstand how customers incorporate Scout into industry-specific workflows, and what a correct answer looks like in each\nSeek out customer conversations proactively and use what you learn to inform decisions\nCan demo your area in depth and give a solid cross-product demo\nPresentation & Communication\nPresent compellingly to senior stakeholders and bring people along through the process\nModel the \"build in public\" culture by sharing work early and often, inviting critique, and showing how work evolves based on input\nTranslate complex technical and quantitative concepts for diverse audiences\nWithin 90 Days, You'll:\nCollaborate regularly with product management, design, and engineering peers, and have met key stakeholders across the departments you work with\nBe on the phone with customers, learning their workflows, pain points, and how they use Sitetracker and Scout. Know the personas and subverticals\nDeliver work to your scrum team - pushing code, building instrumentation and evals, and demonstrating the builder standard\nBe able to confidently demo Sitetracker's core product in your area and give a...","description_format":"text","description_chars":13795,"description_truncated":true,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Project & Work Management Software"],"lifecycle":[{"event":"open","at":"2026-10-15T17:45:53Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 1 · green card filings: 1","filings_12m":1,"filings_prev_12m":7,"green_card_filings_12m":1,"median_offered_wage_usd":115000,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":0}],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":78,"reasons":["conf:0","win:early"],"computed_at":"2026-10-16T01:47:26Z"},"pay":{"stated_usd_annual":215000,"is_top_pay":true},"html_url":"https://alion.io/job/sitetracker-senior-data-scientist-agentic-ux-design","json_url":"https://alion.io/job/sitetracker-senior-data-scientist-agentic-ux-design.json","meta":{"generated_at":"2026-10-16T01:47:26Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2864,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":7358},"rest":"https://alion.io/mcp/rest/get_company?id=7358"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsitetracker-senior-data-scientist-agentic-ux-design"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsitetracker-senior-data-scientist-agentic-ux-design"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsitetracker-senior-data-scientist-agentic-ux-design"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/sitetracker-senior-data-scientist-agentic-ux-design\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsitetracker-senior-data-scientist-agentic-ux-design"}]}