{"id":4352501,"url":"https://alion.io/job/gic-vp-site-reliability-observability-engineering-datadog-arize-technology-group","title":"VP, Site Reliability & Observability Engineering (Datadog, Arize), Technology Group","company":{"id":2761,"name":"GIC","domain":"gic.com.sg","url":"https://alion.io/company/gic","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SuccessFactors","truth_index":null},"role":"Leadership","role_family":"Leadership","seniority":"head","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Singapore"],"countries":["SG"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"AIOps","optional":false},{"name":"Amazon CloudWatch","optional":false},{"name":"Anomaly Detection","optional":false},{"name":"Ansible","optional":false},{"name":"Arize Phoenix","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Chaos Engineering","optional":false},{"name":"CI/CD","optional":false},{"name":"Datadog","optional":false},{"name":"Dynatrace","optional":false},{"name":"Hallucination","optional":false},{"name":"Incident Management","optional":false},{"name":"ITSM","optional":false},{"name":"LangGraph","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Platform Engineering","optional":false},{"name":"Python","optional":false},{"name":"Self-Healing","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"Splunk","optional":false},{"name":"Terraform","optional":false},{"name":"Azure DevOps","optional":true},{"name":"ITIL","optional":true},{"name":"LangChain","optional":true}],"status":"live","first_seen_at":"2026-11-16T03:42:54Z","employer_posted_date":"2026-11-16","last_verified_at":"2026-11-16T03:42:54Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"GIC is one of the world’s largest sovereign wealth funds. With over 2,000 employees across 11 locations around the world, we invest in more than 40 countries globally across asset classes and businesses. Working at GIC gives you exposure to an extraordinary network of the world’s industry leaders. As a leading global long-term investor, we Work at the Point of Impact for Singapore’s financial future, and the communities we invest in worldwide.\nTechnology Group\n We experiment, design, and lead a 24×7 global business where we support core capabilities in asset management, trading, investment operations, and risk management. We deliver secure, reliable, and integrated solutions, and provide insights on new, and emerging technologies.\nInfrastructure & Cybersecurity Resilience (ICR)\nWe design, build, and secure the technology foundations that power GIC’s global investment operations. We aim to deliver resilient, scalable, and secure infrastructure that empowers our people and businesses to perform securely, efficiently, and effectively.\nWhat impact will you make in this role?\nAs a VP, Site Reliability & Observability Engineer you will be the hands-on subject matter expert (SME) who governs, designs, and engineers the enterprise observability estate across infrastructure, applications, and AI/ML systems. You will drive proactive monitoring, automation, and resilience using Datadog, Dynatrace, Arize (ML/LLM observability), and the AWS and Azure platforms - ensuring operational stability, risk control, and compliance with financial regulatory standards.\nThis is a deep hands-on, individual-contributor SME role - not a people-management position. You will personally instrument services, build telemetry pipelines, write automation, and engineer the platform as code. You bring an AI-first mindset, using AI and agentic-AI solutions to operate, investigate, and automate observability - and you also observe the AI itself, monitoring model and LLM behaviour in production. You will define and apply SRE strategy - SLIs, SLOs, and error budgets - and build automation that reduces MTTD and MTTR and strengthens operational resilience.\nWhat will you do as a Site Reliability & Observability Engineer?\nObservability Engineering & Governance\nDesign and own the enterprise observability architecture aligned with operational resilience mandates (MAS TRM, DORA, APRA CPS 230).\nDeploy, instrument, and optimize observability platforms - Datadog, Dynatrace, Splunk - for full-stack visibility (infra, application, network, and user experience), hands-on.\nEstablish telemetry governance for metrics, logs, and traces (incl. OpenTelemetry) - consistency, retention compliance, and security controls - implemented as code.\nIntegrate observability with incident management, ITSM, and AIOps for predictive alerting and anomaly detection.\nAI/ML & LLM Observability (Arize)\nEngineer observability for AI/ML and LLM systems using Arize - monitoring model performance, data drift, feature drift, prediction quality, bias, and LLM response quality in production.\nBuild evaluation, tracing, and guardrail monitoring for LLM and agentic-AI applications (prompt/response tracing, hallucination and toxicity signals, latency and cost).\nDefine SLIs/SLOs and alerting for model and LLM health, and integrate ML observability into the broader reliability and incident-management workflow.\nPartner with Data Science and ML Engineering to close the loop from production signals back into retraining and model governance.\nAI-First Automation & Agentic AI Engineering\nAdopt an AI-first operating model: use AI copilots, LLMs, and agentic-AI workflows (e.g., Python, LangGraph) to operate, investigate, and automate the observability platform through code.\nBuild agentic-AI investigators that correlate telemetry, enrich context, triage alerts, run guided diagnostics, and draft incident timelines and root-cause hypotheses.\nApply AI/ML for anomaly detection, intelligent event correlation and deduplication, alert-noise reduction, and predictive alerting across infrastructure and applications.\nEngineer closed-loop, AI-assisted self-healing and auto-remediation integrated with runbooks and IaC, with explicit guardrails, blast-radius limits, and human-in-the-loop controls for high-risk actions.\nTreat telemetry pipelines, automation, and AI agents as governed products - version-controlled, tested, observable, secure, and auditable.\nReliability Engineering & SRE Strategy\nDefine and apply SRE strategy for infrastructure and business-critical applications - SLIs, SLOs, error-budget policies, and reliability scorecards for key production services.\nDesign and automate runbooks, alerts, self-healing actions, and auto-remediation via Python, Ansible, and Terraform.\nReduce MTTD/MTTR, incident recurrence, and manual intervention through observability- and AI-led automation; track toil reduction explicitly.\nConduct resilience testing, chaos engineering, and capacity validation aligned with business-continuity standards.\nPartner hands-on with Application, Infrastructure, and Cyber teams to codify reliability into the delivery lifecycle.\nCloud Observability & Platform Engineering\nArchitect and manage observability for cloud-native workloads in AWS and Azure (CloudWatch, X-Ray, CloudTrail; Monitor, Log Analytics, App Insights) across compute, storage, and network layers.\nIntegrate cloud observability into landing zones and CI/CD pipelines for continuous compliance with deployment controls.\nImplement infrastructure-as-code using Terraform and Ansible for consistent, auditable provisioning.\nOperational Excellence & Technical Enablement\nServe as the highest escalation point for complex incidents and outages; perform deep-dive RCA and engineer long-term preventive solutions; drive blameless post-mortems.\nPartner with Service Delivery, Cyber, and Application teams for predictive incident prevention and root-cause transparency.\nDeliver executive dashboards highlighting availability, reliability KPIs, and operational-risk indicators.\nCreate reusable instrumentation, patterns, and runbooks; uplift peers on observability, SRE, and AI-assisted automation.\nWhat makes you a successful candidate?\nBachelor’s or Master’s degree in Computer Science, Engineering, or a related discipline.\n12+ years in Infrastructure, Cloud, or SRE roles, with 5+ years hands-on in an SRE/observability SME capacity in financial institutions or regulated environments.\nProven hands-on expertise in:\nObservability Platforms: Datadog, Dynatrace, Splunk, ELK; OpenTelemetry instrumentation.\nAI/ML Observability: Arize or equivalent ML/LLM observability - model/LLM monitoring, drift, and evaluation.\nAutomation / IaC: Terraform, Ansible, Python, and CI/CD tooling.\n AI / Agentic AI: Practical use of LLMs and agentic-AI frameworks (e.g., Python, LangGraph) to build operational agents, investigations, and auto-remediation.\n Cloud Platforms: AWS (CloudWatch, X-Ray, CloudTrail), Azure (Monitor, Log Analytics, App Insights).\nDeep understanding of SRE strategy - service-health modelling, SLIs/SLOs, error budgets, and auto-remediation design.\nProven architectural and design skills for scalable, resilient, high-performance observability platforms.\nStrong analytical and deep-dive troubleshooting skills, with a bias toward long-term preventive, engineered solutions.\nFamiliarity with financial-sector operational resilience frameworks, regulatory compliance, and incident governance.\nCertifications Required:\nDatadog Certified Observability Professional / Dynatrace Certified Associate (or higher)\nTerraform / Ansible / Python certification\nHighly desirable:\nAWS Certified DevOps Engineer / Azure DevOps Engineer Expert\nSRE Foundation / Practitioner (DevOps Institute)\nITIL v4\nKey Responsibility Areas\nArchitecture & Design - Design and evolve highly scalable, resilient, performant observability leveraging Datadog, Dynatrace, Splunk, and Arize; define best practices and patterns as code.\nAI/ML & LLM Observability - Monitor model/LLM health with Arize - drift, quality, bias, hallucination, latency, cost - and integrate into reliability and incident workflows.\n AI-First Automation - Use AI, LLMs, and agentic-AI workflows (Python/LangGraph) to operate and automate the platform via code - investigation, correlation, predictive alerting, and guarded auto-remediation.\n Reliability (SRE) Strategy - Define SLIs/SLOs and error-budget policies; reduce MTTD/MTTR, incident recurrence, and toil through engineered automation.\nTroubleshooting & Incidents - Highest escalation point for complex incidents and outages; deep-dive RCA; drive blameless post-mortems and preventive fixes\nPerformance & Capacity - Identify bottlenecks, optimize resources, and forecast capacity for continued reliability and efficiency.\n Automation & Tooling (IaC) - Build instrumentation, runbooks, and self-healing via Python/Ansible/Terraform following IaC principles - version-controlled and auditable.\nSecurity & Compliance - Enforce telemetry governance, retention, access controls, and audit logging; ensure compliance with MAS TRM, DORA, CPS 230, and internal policy\nWork at the Point of Impact\nWe need to be forward-looking to attract the right people to help us become the Leading Global Long-term Investor. Join our ambitious, agile, and diverse teams - be empowered to push boundaries and pursue innovative ideas, share your views, and be heard. Be anchored on our PRIME Values: Prudence, Respect, Integrity, Merit and Excellence, which guides us in how we make our day-to-day decisions. We strive to inspire. To make an impact.\nGIC is a Great Place to Work\nAt GIC, our offices are vibrant hubs for ideation, professional growth, and interpersonal connection. At the same time, we believe that flexibility allows us to do our best work and be our best selves. Thus, our teams come into the office four days per week to harness the benefits of in-person collaboration, but have the flexibility to choose which days they work from home and adjust this arrangement as situational needs arise.\nGIC is an equal opportunity employer\nAs an employer, we passionately believe every individual brings with them unique diversity of thought and perspectives to meaningfully enrich perspectives of GIC teams to drive competitive performance. An inclusive environment yields exceptional contribution.\nLearn more about our Technology Group here:\nhttps://gic.careers/group/technology-group/","description_format":"text","description_chars":10442,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Pension & Sovereign Wealth Funds"],"lifecycle":[{"event":"open","at":"2026-11-16T03:42:54Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":0,"expected_fill_days":28,"reasons":["conf:2","velocity","win:early","comp:brand"],"computed_at":"2026-11-16T05:46:25Z"},"pay":null,"html_url":"https://alion.io/job/gic-vp-site-reliability-observability-engineering-datadog-arize-technology-group","json_url":"https://alion.io/job/gic-vp-site-reliability-observability-engineering-datadog-arize-technology-group.json","meta":{"generated_at":"2026-11-16T06:16:34Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9110,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2761},"rest":"https://alion.io/mcp/rest/get_company?id=2761"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fgic-vp-site-reliability-observability-engineering-datadog-arize-technology-group"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fgic-vp-site-reliability-observability-engineering-datadog-arize-technology-group"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fgic-vp-site-reliability-observability-engineering-datadog-arize-technology-group"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/gic-vp-site-reliability-observability-engineering-datadog-arize-technology-group\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fgic-vp-site-reliability-observability-engineering-datadog-arize-technology-group"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4352501}},{"tool":"search_jobs","arguments":{"company":"gic.com.sg","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4352501},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}