Senior Engineer - L2 Observability

Presidio, Where Teamwork and Innovation Shape the Future

At Presidio, we're at the forefront of a global technology revolution, transforming industries through cutting-edge digital solutions and next-generation AI. We empower businesses - and their internal customers - to achieve more through innovation, automation, and intelligent insights.

The Role

The Level 2 Managed Service Engineer for the End-to-End Observability team is responsible for advanced troubleshooting, root cause analysis, Datadog configuration, incident escalation, and continual service improvement across the observability stack. This role serves as the primary escalation point for L1 engineers and is expected to independently resolve, or drive resolution of, moderately complex incidents spanning application, infrastructure, network, APM, Synthetics, logs, and RUM data across both 3-tier and cloud-native environments.

Key Responsibilities

• Act as the first escalation point for incidents triaged by L1 engineers that require deeper investigation.
• Perform root cause analysis (RCA) by correlating APM traces, infrastructure metrics, network telemetry, logs, Synthetics results, and RUM data.
• Configure and fine-tune advanced Datadog capabilities: custom metrics, log pipelines, monitors with composite/anomaly detection, SLOs, and dashboards aligned to service ownership.
• Guide and review Datadog APM instrumentation coverage across services; identify monitoring gaps and recommend improvements.
• Independently identify whether an issue originates in the application tier, infrastructure, network, or a third-party dependency, in both 3-tier and cloud-native (microservices/containerized) environments.
• Lead or actively participate in major incident bridge calls, providing observability data to speed up resolution.
• Own problem management: identify recurring incidents, drive permanent fixes, and track them to closure.
• Mentor and coach L1 engineers on triage techniques, Datadog usage, and escalation quality.
• Contribute to automation of routine monitoring tasks (alert enrichment, auto-remediation triggers, ticket auto-creation).
• Support capacity and performance trend analysis using historical Datadog data.
• Review and refine monitors to reduce noise and false positives (alert tuning).
• Ensure adherence to SLAs/OLAs and maintain quality of incident documentation and RCA reports.
• Participate in client/stakeholder calls to explain incidents, trends, and improvement plans.

Required Skills and Experience

Observability Platform Skills

• 3 to 6 years of experience in Managed Services, NOC, SRE, or application/infrastructure support roles, with at least 1-2 years working hands-on with Datadog.
• Strong working knowledge of Datadog across monitoring, dashboards, alerting, APM, Synthetics, RUM, logs, metrics, SLOs, and service-level troubleshooting.
• Advanced monitor configuration (anomaly detection, forecast, composite monitors, SLOs).
• APM: distributed tracing, service maps, trace analytics, and identifying instrumentation gaps.
• Synthetics: designing multi-step API and browser tests, analyzing failure trends.
• RUM: correlating front-end performance and errors with backend traces.
• Log Management: building log pipelines, parsing rules, and log-based metrics.
• Solid understanding of infrastructure concepts: compute, storage, containers, orchestration (Kubernetes basics), and cloud resource scaling.
• Solid understanding of networking fundamentals: DNS, load balancing, latency/packet loss analysis, firewall and connectivity troubleshooting.
• Strong analytical ability to read application logs, stack traces, and error patterns to isolate the failing component, without needing to be an expert in the underlying code.
• Experience with ITSM tools for incident, problem, and change management.
• Working knowledge of ITIL processes, especially incident and problem management.

Application and Architecture Understanding

• Solid understanding of 3-tier application architecture and how load, latency, or failure at one tier cascades to others.
• Solid understanding of cloud-native architecture: microservices, containers, API gateways, service meshes, and inter-service communication patterns.
• Ability to correlate application-level symptoms (errors, latency, timeouts) with underlying infrastructure or network causes using observability data alone.
• Understanding of common resiliency patterns (retries, circuit breakers, caching, load balancing) and how their failure shows up in monitoring data.

Good to Have

• Datadog certification (APM, Infrastructure, or Log Management specialization).
• Scripting experience (Python, Shell, PowerShell) for automation and custom integrations.
• Exposure to Infrastructure as Code (Terraform) for managing Datadog configuration as code.
• Experience with CI/CD pipelines and how deployments correlate with monitoring anomalies.
• Exposure to other observability or APM tools (New Relic, Dynatrace, Splunk, Grafana, Prometheus).

Soft Skills

• Strong ownership and accountability for incident resolution.
• Ability to communicate technical findings clearly to both technical teams and stakeholders.
• Mentoring ability and patience when guiding L1 engineers.
• Comfortable leading discussions during high-pressure major incidents.
• Structured approach to documentation and knowledge sharing.

Education

• Bachelor's degree in Computer Science, Information Technology, Electronics, or a related field (or equivalent practical experience).

Work Details

• Level: L2 (Mid-level, Managed Services)
• Shift: 24x7 rotational shifts, with on-call/escalation coverage as needed
• Reporting To: L3 Engineer / Observability Team Lead

Your future at Presidio

Joining Presidio means stepping into a culture of trailblazers - thinkers, builders, and collaborators - who push the boundaries of what's possible. With our expertise AI-driven analytics, cloud solutions, cybersecurity, and next-gen infrastructure, we enable businesses to stay ahead in an ever-evolving digital world.

Here, your impact is real. Whether you're harnessing the power of Generative AI, architecting resilient digital ecosystems, or driving data-driven transformation, you'll be part of a team that is shaping the future.

Ready to innovate? Let's redefine what's next-together.

About Presidio

Presidio is committed to hiring the most qualified candidates to join our amazing culture. We aim to attract and hire top talent from all backgrounds, including underrepresented and marginalized communities. We encourage women, people of color, people with disabilities, and veterans to apply for open roles at Presidio. Diversity of skills and thought is a key component to our business success.

At Presidio, speed and quality meet technology and innovation. Presidio is a trusted ally for organizations across industries with a decades-long history of building traditional IT foundations and deep expertise in AI and automation, security, networking, digital transformation, and cloud computing. Presidio fills gaps, removes hurdles, optimizes costs, and reduces risk. Presidio's expert technical team develops custom applications, provides managed services, and enables actionable data insights and builds forward-thinking solutions that drive strategic outcomes for clients globally. For more information visit https://presidio.com

Applications will be accepted on a rolling basis.

Presidio has a strong commitment to the community we serve and our employees. As an Equal Opportunity Employer, we strive to have a workforce that includes the community we serve.

Recruitment Agencies, Please Note: Presidio does not accept unsolicited agency resumes/CVs. Do not forward resumes/CVs to our career's email address, Presidio employees or any other means. Presidio is not responsible for any feeds related to unsolicited resumes/CVs.