{"id":3308664,"url":"https://alion.io/job/chegg-senior-software-engineer-ai-evals","title":"Senior Software Engineer - AI Evals","company":{"id":20322,"name":"Chegg","domain":"chegg.com","url":"https://alion.io/company/chegg","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Backend","role_family":"Backend","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Delhi, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":17000,"max_usd":40000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":168},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Amazon SageMaker","optional":false},{"name":"AWS","optional":false},{"name":"AWS Bedrock","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"DPO","optional":false},{"name":"Fine-tuning","optional":false},{"name":"Function Calling","optional":false},{"name":"Hallucination","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"LLM","optional":false},{"name":"MLFlow","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Reward Modeling","optional":false},{"name":"RLAIF","optional":false},{"name":"RLHF","optional":false},{"name":"SFT","optional":false},{"name":"TensorFlow","optional":false},{"name":"Post-training","optional":true}],"status":"live","first_seen_at":"2026-10-27T00:24:05Z","employer_posted_date":null,"last_verified_at":"2026-10-27T00:24:05Z","board_verified":false,"closed_at":null,"days_open":6,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":6},"description":"Job Description\n\nAbout the Role\n\nChegg is building the next generation AI Evaluation Platform to keep improving learner experience across product and services from copilots to autonomous agents. As we scale our use of large language models and agentic systems, the quality, safety, and reliability of these systems depend on rigorous, well-designed evaluations. We're hiring a Senior Engineer to own the design and implementation of evaluation frameworks (evals) that measure model and agent performance across accuracy, reasoning, safety, and pedagogical quality — and that directly inform how we train, fine-tune, and ship AI systems.\n\nThis is a high-leverage, cross-functional role sitting at the intersection of ML research, data engineering, and product. You'll own evaluation solutions end-to-end — from framework design through deployment and production monitoring — partnering closely with ML researchers, applied scientists, and product teams to define what good looks like for models and agents and turn measurements into actionable outcomes.\n\nWhat You'll Do\n\nOwn evaluation solutions end-to-end: design the methodology, build the pipeline and tooling, deploy it into training and production workflows, and maintain/iterate on it over time — with minimal hand-off to other teams.\nDesign and build evaluation frameworks and harnesses for LLMs and multi-step AI agents, covering offline benchmarks, online/production evals, and human-in-the-loop review.\nDefine rubrics and scoring methodologies for open-ended tasks (e.g., tutoring quality, step-by-step reasoning, citation accuracy) where correctness isn't binary.\nBuild automated, model-graded (LLM-as-judge) and rule-based evaluators, and validate them against human judgment for reliability and bias.\nDevelop agent-specific evaluations: tool-use correctness, multi-turn task completion, planning/trajectory quality, failure recovery, and cost/latency tradeoffs.\nCreate golden datasets, adversarial test sets, and regression suites that catch quality and safety regressions before they reach production.\nPartner with research teams to translate eval results into training signal — informing SFT/RLHF/RLAIF data curation, reward modeling, and fine-tuning priorities.\nInstrument production systems to collect real-world interaction data and feed it back into the evaluation and training loop.\nBuild dashboards and reporting that give researchers, PMs, and leadership a clear, trustworthy view of model quality trends across releases.\nDrive eval methodology rigor: statistical significance, inter-rater reliability, sampling strategy, and avoiding metric gaming or overfitting to benchmarks.\nCollaborate with Trust & Safety and Legal/Compliance stakeholders to build evals for bias, hallucination, academic integrity, and other responsible-AI dimensions relevant to an education product.\nMentor other engineers on eval best practices and help establish evaluation as a first-class part of the model development lifecycle.\n\nWhat We're Looking For\n\n5+ years of software/ML engineering experience, including hands-on work building or maintaining evaluation, testing, or measurement infrastructure for ML systems.\nDirect experience with foundation model evaluation and benchmarking beyond using APIs — experience gained at a foundation model lab or similarly frontier research environment is strongly preferred.\nDemonstrated ability to own an evaluation solution end-to-end — from initial design and dataset/methodology creation through pipeline build, deployment, and production monitoring — with minimal hand-off.\nDirect experience designing evals for LLMs and/or AI agents — e.g., benchmark design, LLM-as-judge pipelines, human annotation platforms, or A/B and offline/online eval frameworks.\nStrong programming skills in Python, PyTorch / Tensorflow and experience building production-grade data/ML pipelines.\nHands-on knowledge of key model training and evaluation platforms, particularly AWS (e.g., SageMaker, Bedrock) and Databricks (e.g., MLflow, Unity Catalog, Delta Lake).\nSolid grounding in applied statistics — comfortable reasoning about sample size, variance, significance testing, and the limitations of aggregate metrics.\nWorking knowledge of how LLMs are trained and adapted (pretraining, SFT, RLHF/RLAIF/DPO) and how eval signal feeds into that lifecycle.\nExperience with agentic architectures — tool calling, multi-step planning, memory, orchestration frameworks — and the unique evaluation challenges they introduce.\nFamiliarity with eval and observability tooling (e.g., internal or open-source frameworks for tracing, dataset versioning, experiment tracking).\nExcellent cross-functional collaboration skills; able to translate ambiguous product/research questions into concrete, measurable eval criteria.\nA bias toward rigor and skepticism — you instinctively question whether a metric is actually measuring what it claims to.\n\nBonus Points\n\nWe prioritize candidates with direct experience in the following areas:\n\nPost-training & evals: hands-on experience with post-training techniques (SFT, RLHF, RLAIF, DPO) and the evaluation methodologies used to validate them, ideally gained at a frontier model labs\nModel benchmarking: experience building, running, or maintaining benchmark suites used to track and compare frontier model capabilities across training runs and releases.\nData quality: experience with the data quality side of model training — curation, filtering, deduplication, and quality scoring of pretraining and post-training datasets.\n\nWhy Chegg\n\nYou'll shape how Chegg measures and improves the AI systems millions of students rely on — with direct influence on model training decisions, product quality, and responsible AI practices. This role offers high visibility across research, engineering, and product leadership, and the opportunity to help define evaluation as a discipline within the company's AI strategy.\n\nCompensation & Benefits\n\nSalary range and benefits will be shared per Chegg's compensation bands and the candidate's location, in accordance with applicable pay transparency requirements.\n\nChegg is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.\n\nWhy do we exist\n\nStudents are working harder than ever before to stabilize their future. Our recent research study called State of the Student shows that nearly 3 out of 4 students are working to support themselves through college and 1 in 3 students feel pressure to spend more than they can afford. We founded our business on provided affordable textbook rental options to address these issues. Since then, we've expanded our offerings to supplement many facets of higher educational learning through Chegg Study, Chegg Math, Chegg Writing, Chegg Internships, Chegg Skills, and more to support students beyond their college experience. These offerings lower financial concerns for students by modernizing their learning experience. We exist so students everywhere have a smarter, faster, more affordable way to student.\n\nVideo Shorts\n\nLife at Chegg: http://youtu.be/Fwf90zgaOLA\n\nChegg Corporate Career Page: https://jobs.chegg.com/\n\nChegg India: http://www.cheggindia.com/\n\nChegg Israel: http://www.chegg.com/about/working-at-chegg/israel/\n\nChegg Skills: https://www.chegg.com/skills\n\nChegg out our culture and benefits!\n\nhttp://www.chegg.com/about/working-at-chegg/benefits/\n\nChegg is an equal opportunity employer","description_format":"text","description_chars":7456,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence"],"lifecycle":[{"event":"open","at":"2026-10-27T02:09:45Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":6,"expected_fill_days":28,"reasons":["seen:6","win:early"],"computed_at":"2026-11-02T06:13:15Z"},"pay":null,"html_url":"https://alion.io/job/chegg-senior-software-engineer-ai-evals","json_url":"https://alion.io/job/chegg-senior-software-engineer-ai-evals.json","meta":{"generated_at":"2026-11-02T22:26:03Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":31534,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":20322},"rest":"https://alion.io/mcp/rest/get_company?id=20322"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fchegg-senior-software-engineer-ai-evals"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fchegg-senior-software-engineer-ai-evals"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fchegg-senior-software-engineer-ai-evals"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/chegg-senior-software-engineer-ai-evals\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fchegg-senior-software-engineer-ai-evals"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3308664}},{"tool":"search_jobs","arguments":{"company":"chegg.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3308664},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}