{"id":4199047,"url":"https://alion.io/job/falls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2","title":"Machine Learning Engineer / SME (LLM Eval & RAG Testing) – SME | TS/SCI","company":{"id":4483010,"name":"Falls Technology","domain":"fallstechnology.com","url":"https://alion.io/company/falls-technology","size_band":null,"is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Schema","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["McLean, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"GitHub","optional":false},{"name":"Jenkins","optional":false},{"name":"Keras","optional":false},{"name":"LLM","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"RAG","optional":false},{"name":"SQL","optional":false},{"name":"TensorFlow","optional":false},{"name":"Explainable AI","optional":true},{"name":"Flask","optional":true},{"name":"Hugging Face","optional":true},{"name":"ONNX","optional":true},{"name":"Scikit-learn","optional":true},{"name":"Semantic Search","optional":true},{"name":"Tableau","optional":true},{"name":"Transformers","optional":true}],"status":"live","first_seen_at":"2026-10-28T00:00:00Z","employer_posted_date":"2026-10-28","last_verified_at":"2026-11-18T07:03:12Z","board_verified":true,"closed_at":null,"days_open":21,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":21},"description":"Active TS/SCI required. U.S. citizenship required.\nLocation: On-site in McLean, VA (Full-Time).\nProve which LLMs actually work for the mission. You'll design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks so the team ships AI systems they can trust.\nWhat you'll do\nTest, evaluate and validate AI systems with a focus on LLM-based systems (including RAG testing)\nDesign and run experiments that benchmark LLMs against defined measures\nTune neural networks / LLMs on custom datasets and apply results to concrete use cases\nBuild AI/ML models and ensembles in Python or R (library selection, preprocessing, model assessment)\nUse deep learning frameworks such as PyTorch, TensorFlow or Keras\nLeverage GPUs for accelerated training and evaluation\nWrite solid SQL (CTEs, set operations, aggregates, nested subqueries) against analytical datasets\nWork in GitHub and Jenkins-style CI for reproducible experiments\nCommunicate methodological choices and model results to technical and non-technical stakeholders\nWhat you bring (required)\nActive TS/SCI\nHands-on RAG testing experience (must-have)\nTuning neural nets / LLMs on custom datasets for real use cases\nBuilding AI/ML models and ensembles in Python or R\nPyTorch, TensorFlow or Keras (key)\nGPU-accelerated computing (key)\nTesting, evaluating or validating LLM-based AI systems (key)\nExperiment design to benchmark LLMs (key)\nSQL including CTEs, set ops, aggregates and nested subqueries\nGitHub and Jenkins (or equivalent) version control / CI\nNice to have\nHugging Face Transformers library and hub\nBuilding benchmarks that measure LLM performance\nCloud computing development and architecture\nFlask or similar front-end/web frameworks\nText classification and topic modeling (scikit-learn or deep learning)\nSemantic search applications\nTableau dashboards\nVerification/validation test benches, Explainable AI (XAI), ONNX\nWhy Falls\nA 9% retirement contribution, 100% of family medical premiums paid, and 4 weeks of PTO. We're a small business that's been delivering since 2019, so your work is visible and your voice counts.\nFalls Technology is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, national origin, disability, or protected veteran status.","description_format":"text","description_chars":2325,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":true,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Systems Integrators","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-12T21:31:02Z"}],"visa":[],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.835,"p_room":0.75,"age_days":21,"expected_fill_days":26,"reasons":["conf:4","velocity","win:late"],"computed_at":"2026-11-18T05:48:20Z"},"pay":null,"html_url":"https://alion.io/job/falls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2","json_url":"https://alion.io/job/falls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2.json","meta":{"generated_at":"2026-11-18T08:33:57Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":15008,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4483010},"rest":"https://alion.io/mcp/rest/get_company?id=4483010"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ffalls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ffalls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ffalls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/falls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ffalls-technology-machine-learning-engineer-sme-llm-eval-rag-testing-sme-tssci-2"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4199047}},{"tool":"search_jobs","arguments":{"company":"fallstechnology.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4199047},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}