{"id":1453316,"url":"https://alion.io/job/singtel-senior-agent-operations-evaluation-engineer-aida","title":"Senior Agent Operations & Evaluation Engineer #AIDA","company":{"id":49912,"name":"Singtel","domain":"singtel.com","url":"https://alion.io/company/singtel","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SuccessFactors","truth_index":null},"role":"Industrial Engineering","role_family":"Industrial Engineering","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Singapore"],"countries":["SG"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":40000,"max_usd":85000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":239},"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Incident Management","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Python","optional":false}],"status":"live","first_seen_at":"2026-09-29T08:34:50Z","employer_posted_date":"2026-09-29","last_verified_at":"2026-09-29T21:34:40Z","board_verified":true,"closed_at":null,"days_open":1,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":1},"description":"Powering the Future with AIDA\nTo lead the next phase of our AI evolution, we’ve launched a new business unit AIDA - Artificial Intelligence & Data Analytics - a strategic engine driving our transformation designed to scale our AI ambitions with precision and purpose. This marks a pivotal shift in how we operate, innovate, and serve to embed intelligence into every layer of our business.\nAt Singtel, this is more than a technology upgrade. It’s a strategic transformation that redefines how value is created across the enterprise core-augmenting human capabilities and unlocking entirely new potential. It is a transformation journey by aligning people, platforms, and processes under one cohesive strategy. Our mission is to build AI literacy and foster a culture where intelligence empowers people.\nWe welcome you to join us on a transformational journey that’s reshaping the telecommunications industry - and redefining what’s possible with AI at its core. Grow with us in a workplace that champions innovation, embraces agility, and puts human potential at the heart of everything we do.\nAbout the Role:\nAn engineering role that operates the live GenAI agents and builds the evaluation and observability that proves they are accurate, safe and improving.\nOn the operations side, keeps agents healthy, performs first-level triage and incident response, and minimises downtime.\nOn the evaluation and observability side, builds the operational dashboard for each use case, defines the metrics that matter, runs the evaluation suites, and proactively detects issues before customers do.\nWorks closely with the Day 1 build teams, solution architects and business owners across Singtel Singapore.\nHow You will Make An Impact:\nBuild and maintain the operational observability dashboard for each AI / agent and Classical AI ML use case, across system health, performance, quality, safety and business-outcome metrics.\nDefine, with build teams and solution architects (technical metrics) and product / business owners (outcome metrics), what needs to be measured per use case; consume the telemetry the platform provides.\nBuild and run the evaluation suites (offline before launch and continuous evaluation on sampled production traffic after) and drive proactive detection of drift and quality decay.\nMonitor live agents, using the dashboards and automated safeguards, and act on alerts and proactive signals.\nPerform first-level triage and in-scope fixes (restart, safe-listed configuration, cache reset, routing fallback) per the runbooks.\nManage incidents - classify severity, route by fault domain, escalate with a full evidence bundle, and verify-and-close in production - while owning the incident throughout.\nContribute to the handover gate - verifying operability and evaluation evidence before an agent is accepted into full operation.\nSkills for Success:\nDegree in Computer Science, Engineering, Data Science, or a related field\n6 - 10 years across software / SRE / data or ML engineering\nHands-on production operations experience AND hands-on evaluation, data-analysis or quality experience\nExperience with LLM / agentic systems\nMonitoring, alerting and observability; log / trace analysis\nEvaluation methods and metric design for AI systems\nDashboarding tools and Python\nUnderstanding of LLM / agent architectures and failure modes\nLLM observability and evaluation tooling (e.g. tracing, eval harnesses)\nIncident management practice\nComposure and clear communication under incident pressure\nAnalytical rigour and attention to detail\nAbility to context-switch and prioritise across two disciplines\nClear technical writing for gate findings\nAbility to influence build teams on quality\nUnderstanding of telco customer intents and journeys\nAre you ready to say hello to BIG Possibilities?\nJoin Singtel to shape what's next and accelerate your career through meaningful work, continuous learning, and real impact.","description_format":"text","description_chars":3918,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Telecommunications","Broadband"],"lifecycle":[{"event":"open","at":"2026-09-29T08:34:50Z"}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":0,"expected_fill_days":14,"reasons":["conf:8","velocity","win:early","comp:brand"],"computed_at":"2026-09-30T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/singtel-senior-agent-operations-evaluation-engineer-aida","json_url":"https://alion.io/job/singtel-senior-agent-operations-evaluation-engineer-aida.json","meta":{"generated_at":"2026-10-01T03:11:09Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","usage":{"tier":"crawler","counted_by":"address","units_charged":1,"used_today":2678,"day_limit":5000,"remaining_today":2322,"minute_limit":60,"resets_at":"2026-10-02T00:00:00Z"}}}