{"id":4190826,"url":"https://alion.io/job/bask-health-ml-platform-engineer-evaluation-systems","title":"ML Platform Engineer, Evaluation Systems","company":{"id":66537,"name":"Bask Health","domain":"bask.health","url":"https://alion.io/company/bask-health","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":{"grade":"A","score":88,"open_postings":21,"ghost_share":0,"stale_share":0.286,"repost_share":0,"time_to_fill_p50_days":171,"computed_at":"2026-11-15T07:52:55Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"CI/CD","optional":false},{"name":"Claude Code","optional":false},{"name":"Cursor","optional":false},{"name":"Git","optional":false},{"name":"Machine Learning","optional":false},{"name":"Platform Engineering","optional":false},{"name":"Python","optional":false},{"name":"TypeScript","optional":false},{"name":"Amazon EventBridge","optional":true},{"name":"Amazon SageMaker","optional":true},{"name":"AWS","optional":true},{"name":"AWS Lambda","optional":true},{"name":"Dagster","optional":true},{"name":"DeepEval","optional":true},{"name":"Docker","optional":true},{"name":"HIPAA","optional":true},{"name":"Kubernetes","optional":true},{"name":"LangSmith","optional":true},{"name":"MLFlow","optional":true},{"name":"Ragas","optional":true},{"name":"Weights & Biases","optional":true}],"status":"live","first_seen_at":"2026-11-12T19:17:00Z","employer_posted_date":"2026-11-12","last_verified_at":"2026-11-15T03:21:16Z","board_verified":true,"closed_at":null,"days_open":2,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":2},"description":"About Bask Health\nBask Health is a software platform that enables entrepreneurs, doctors, and healthcare businesses to launch, manage, and grow direct-to-consumer telehealth businesses.\nOur platform brings together patient experiences, clinical workflows, pharmacy operations, and commerce tools in one place.\nWe're an entrepreneurial, product-focused engineering team that values ownership, technical excellence, and building software that solves real problems. Engineers work closely with product teams and company leadership, take features from idea to production, and have a direct role in shaping the platform.\nEngineering at Bask is AI-first and builder-led. We use tools like Cursor and Claude Code throughout development while maintaining high standards for reliability, testing, and software quality.\nAbout the Role\nWe're looking for an ML Platform Engineer, Evaluation Systems to build and maintain the infrastructure that allows Bask Health to evaluate machine learning models and AI-powered applications reliably, consistently, and at scale.\nBask Health supports direct-to-consumer telehealth businesses through a platform that connects patient experiences, merchant operations, clinical workflows, pharmacy coordination, and commerce capabilities. As machine learning and AI become more integrated into these systems, we need dependable infrastructure for measuring model quality, comparing changes, and understanding how AI systems perform before and after deployment.\nYou'll help develop the platform services, data pipelines, execution environments, and monitoring capabilities that support machine learning evaluation across engineering teams.\nThis is a hands-on engineering role focused on the operational infrastructure behind evaluation systems. You'll work with cloud services, distributed processing, model integrations, databases, and automated workflows to support repeatable experiments and large-scale evaluation jobs.\nYour work may include building evaluation job orchestration, managing benchmark datasets, developing experiment tracking systems, supporting model comparisons, and creating infrastructure that stores and analyzes evaluation results.\nYou'll collaborate with machine learning engineers, applied scientists, AI evaluation engineers, and backend engineers to understand their technical requirements and build reusable systems that support different model types and evaluation methodologies.\nYou'll also help improve the reliability and reproducibility of evaluation workflows. This includes ensuring experiments can be repeated, datasets and configurations are versioned, failures are observable, and results can be compared accurately over time.\nBecause Bask operates in healthcare-related environments, evaluation infrastructure must support appropriate privacy controls, secure data access, and responsible handling of sensitive information. You'll help ensure evaluation datasets and model outputs are managed according to approved security and operational requirements.\nThis position emphasizes strong infrastructure engineering, practical machine learning knowledge, and ownership of production platform services. You'll contribute to architecture, build scalable systems, improve operational performance, and help engineering teams evaluate AI capabilities more effectively.\nWe're looking for an engineer who enjoys building infrastructure for machine learning teams, understands the challenges of reproducible experimentation, and wants to make AI evaluation more dependable and efficient across the organization.\nWhat You'll Do\nDesign, build, and maintain machine learning evaluation infrastructure across Bask Health.\nDevelop platform services that execute and manage automated model evaluation workloads.\nBuild scalable systems for running large batches of model and AI application evaluations.\nImplement job orchestration, scheduling, and execution management for evaluation pipelines.\nDevelop infrastructure for storing and versioning evaluation datasets.\nBuild systems for tracking model configurations, prompts, parameters, and evaluation results.\nImplement reproducible experiment workflows that support consistent model comparisons.\nDevelop automated pipelines for benchmarking machine learning models.\nBuild APIs and services that allow engineering teams to submit and monitor evaluation jobs.\nCreate infrastructure for comparing evaluation results across model versions and experiments.\nSupport evaluation workflows for large language models, predictive models, and retrieval systems.\nDevelop asynchronous processing systems for computationally intensive evaluation tasks.\nImplement monitoring for evaluation job performance, failures, latency, and resource usage.\nBuild logging and tracing capabilities that help engineers investigate evaluation problems.\nDevelop systems for managing evaluation artifacts, reports, and historical results.\nIntegrate evaluation infrastructure with model registries and ML development workflows.\nImplement automated quality checks that support controlled model releases.\nBuild reliable retry, timeout, and failure recovery mechanisms for evaluation workloads.\nOptimize evaluation execution performance, infrastructure utilization, and operational costs.\nSupport distributed evaluation workloads across cloud-based compute resources.\nCollaborate with machine learning engineers to define evaluation infrastructure requirements.\nWork with applied scientists to support reproducible experiments and benchmark comparisons.\nDevelop tools that simplify evaluation configuration and execution for engineering teams.\nImplement secure access controls for evaluation datasets and results.\nEnsure evaluation systems respect tenant boundaries and approved data-handling practices.\nInvestigate infrastructure incidents and improve evaluation platform reliability.\nDocument evaluation infrastructure, deployment procedures, and operational standards.\nOwn assigned ML evaluation platform projects from technical design through deployment, monitoring, and continuous improvement.\nRequirements\n3+ years of professional experience in software engineering, machine learning engineering, platform engineering, or a closely related technical field.\nExperience building and maintaining production backend services or infrastructure.\nStrong programming skills in Python, TypeScript, or comparable languages.\nFamiliarity with machine learning development and evaluation workflows.\nExperience developing APIs and backend service integrations.\nUnderstanding of distributed systems and asynchronous processing.\nExperience working with cloud-based infrastructure and production deployments.\nFamiliarity with automated testing and continuous integration practices.\nUnderstanding of machine learning model versioning and experiment reproducibility.\nExperience working with relational databases and structured datasets.\nFamiliarity with data pipelines and batch processing workflows.\nExperience designing reliable background job systems.\nUnderstanding of infrastructure scalability, performance, and fault tolerance.\nFamiliarity with logging, monitoring, and production observability.\nExperience implementing retries, timeouts, and error recovery.\nUnderstanding of authentication, authorization, and secure service communication.\nFamiliarity with configuration management and deployment automation.\nExperience writing maintainable, well-tested production code.\nAbility to troubleshoot complex backend or infrastructure issues.\nUnderstanding of software release processes and operational reliability.\nExperience using Git and collaborative engineering workflows.\nAbility to translate machine learning requirements into scalable infrastructure.\nStrong analytical thinking and problem-solving skills.\nAbility to collaborate effectively with machine learning engineers, applied scientists, and software engineers.\nAbility to independently own engineering projects from design through deployment.\nNice to Have\nExperience building infrastructure for machine learning evaluation or experimentation.\nFamiliarity with MLflow, Weights & Biases, or comparable experiment tracking platforms.\nExperience with model registries and artifact management systems.\nFamiliarity with large language model evaluation frameworks.\nExperience with LangSmith, DeepEval, Ragas, or comparable AI evaluation tools.\nExperience building distributed evaluation or benchmarking pipelines.\nFamiliarity with workflow orchestration tools such as Airflow, Dagster, or Temporal.\nExperience with containerized workloads and distributed compute infrastructure.\nFamiliarity with Docker, Kubernetes, or comparable technologies.\nExperience working with AWS Lambda, SQS, EventBridge, or related cloud services.\nFamiliarity with AWS SageMaker or comparable ML platform services.\nExperience with serverless infrastructure and infrastructure-as-code tools.\nFamiliarity with SST or comparable application deployment frameworks.\nExperience developing data versioning and dataset management systems.\nUnderstanding of reproducible machine learning experimentation.\nExperience implementing model performance monitoring and drift detection.\nFamiliarity with large language model APIs and model comparison workflows.\nExperience building internal developer platforms for machine learning teams.\nUnderstanding of HIPAA and healthcare data privacy considerations.\nExperience using AI-assisted development tools such as Cursor or Claude Code.","description_format":"text","description_chars":9446,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Health Care","Telemedicine & Virtual Care","Pharmacies"],"lifecycle":[{"event":"open","at":"2026-11-12T19:17:00Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 3","filings_12m":3,"filings_prev_12m":2,"green_card_filings_12m":0,"median_offered_wage_usd":94167,"route":null,"cap_exempt":false,"checked_at":"2026-11-15T09:21:55+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":0}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":2,"expected_fill_days":171,"reasons":["conf:4","velocity","win:early","comp:brand"],"computed_at":"2026-11-15T07:52:55Z"},"pay":null,"html_url":"https://alion.io/job/bask-health-ml-platform-engineer-evaluation-systems","json_url":"https://alion.io/job/bask-health-ml-platform-engineer-evaluation-systems.json","meta":{"generated_at":"2026-11-15T13:35:16Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4345,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-16T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":66537},"rest":"https://alion.io/mcp/rest/get_company?id=66537"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbask-health-ml-platform-engineer-evaluation-systems"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbask-health-ml-platform-engineer-evaluation-systems"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbask-health-ml-platform-engineer-evaluation-systems"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/bask-health-ml-platform-engineer-evaluation-systems\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbask-health-ml-platform-engineer-evaluation-systems"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4190826}},{"tool":"search_jobs","arguments":{"company":"bask.health","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4190826},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}