{"id":4386345,"url":"https://alion.io/job/wits-innovation-lab-sre-engineer","title":"SRE Engineer","company":{"id":49824,"name":"Wits Innovation Lab","domain":"thewitslab.com","url":"https://alion.io/company/wits-innovation-lab","size_band":null,"is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"middle","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India","Mohali, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"Bash","optional":false},{"name":"DNS","optional":false},{"name":"Docker","optional":false},{"name":"Git","optional":false},{"name":"Grafana","optional":false},{"name":"Incident Management","optional":false},{"name":"Java","optional":false},{"name":"KeyDB","optional":false},{"name":"Kubernetes","optional":false},{"name":"Linux","optional":false},{"name":"OpenSearch","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"TCP/IP","optional":false},{"name":"Ubuntu","optional":false},{"name":"Vector","optional":false},{"name":"VictoriaMetrics","optional":false}],"status":"live","first_seen_at":"2026-11-16T17:35:00Z","employer_posted_date":null,"last_verified_at":"2026-11-16T17:35:00Z","board_verified":false,"closed_at":null,"days_open":0,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":0},"description":"The core responsibilities for the job include the following:\n\nIngestion Operations and Pipeline Management: \nOperate, monitor, and maintain the telemetry ingestion stack across multiple data center sites, including Node Exporter, JMX Exporter, OpenTelemetry agents and collectors, Vector agents and aggregators, SNMP, Blackbox, and syslog collection.\nManage Vector pipelines, including file sources, multiline parsing, VRL transformations, disk buffering, and sink health, ensuring reliable log collection during service restarts and log rotation.\nManage Apache Kafka streaming infrastructure, including topic configurations, partitions, retention, replication health, and consumer-group offsets.\nTroubleshoot consumer lag, stalled consumers, and ingestion failures while ensuring minimal telemetry loss.\nMaintain consistent telemetry labels and resource attributes to enable correlation across metrics, logs, and traces.\n\nOnboarding, Upgrades, and Data Validation: \nDeploy, configure, and upgrade telemetry agents across application, Kafka, Cassandra, KeyDB, PostgreSQL, network, and security hosts.\nValidate network and security requirements, including routing, firewall rules, port connectivity, DNS resolution, and TLS certificates, before onboarding.\nPerform regular health checks, capacity reviews, certificate renewals, patching, and version upgrades.\nValidate configurations and rollback procedures in lower environments before production deployment.\nVerify telemetry completeness after every change and troubleshoot data gaps, and count mismatches and cardinality issues.\n\nMonitoring, Incident Management, and Security: \nMonitor ingestion health through scrape success rates, ingestion volumes, consumer-group lag, buffer utilization, queue depth, and data freshness.\nIdentify and resolve silent source loss, missing telemetry, and pipeline failures.\nBuild and maintain Grafana dashboards and alert rules for infrastructure and ingestion monitoring.\nRespond to incidents within defined SLA timelines, perform root cause analysis, and escalate unresolved issues through established escalation procedures.\nEnsure secure collection endpoints by maintaining appropriate OTLP receiver bindings, read-only log mounts, and preventing unauthorized access to exposed ingestion endpoints.\n\nAutomation and Documentation: \nAutomate operational activities such as agent deployments, configuration management, verification checks, log analysis, and shift reporting using Bash and Python.\nDevelop and maintain operational runbooks, Standard Operating Procedures (SOPs), onboarding guides, and troubleshooting documentation.\nMaintain incident records, RCA reports, and shift handover documentation.\nIdentify opportunities to improve operational efficiency and reduce manual intervention.\n\n24×7 Support & SLA Management: \nProvide continuous operational support across morning, evening, and night shifts, including weekends and public holidays.\nEnsure structured shift handovers covering open incidents, ingestion health, suppressed alerts, and ongoing activities. All handovers must be documented.\nFollow defined incident response and escalation timelines:\nP1 Incidents: Acknowledge within 5 minutes. Automatic escalation after 10 minutes if unacknowledged, with further escalation up to the Incident Commander at 30 minutes.\nP2 Incidents: Acknowledge within 15 minutes, with automatic escalation after 30 minutes.\nParticipate in structured shift overlap and backfill arrangements to ensure uninterrupted operational coverage during planned and unplanned absences.\n\nRequirements: \nExperience: 3+ Years.\nLinux Administration: Strong experience with RHEL/Ubuntu, command-line operations, systemd, networking, and performance troubleshooting.\nOpenTelemetry (OTel): Hands-on experience with Java agent instrumentation, Collector pipelines, OTLP, receivers, processors, and exporters.\nVector.dev : Experience with file sources, multiline parsing, VRL transforms, disk buffering, and sinks.\nPrometheus Ecosystem: Knowledge of Node Exporter, JMX Exporter, SNMP Exporter, Blackbox Exporter, and PromQL.\nPrometheus / VictoriaMetrics: Experience with scrape configurations, relabeling, and remote write.\nLog & Trace Management: Experience with VictoriaLogs (LogsQL) or OpenSearch, along with exposure to trace storage systems.\nApache Kafka: Understanding of topics, partitions, consumer groups, offsets, lag and retention.\nContainerisation: Hands-on experience with Docker and working knowledge of Kubernetes.\nGrafana: Ability to build dashboards, configure panel queries and set up alert rules.\nScripting & Automation: Proficiency in Bash and Python for operational automation.\nNetworking: Strong understanding of TCP/IP, routing, DNS, TLS, firewall rules and port analysis.\nVersion Control: Working knowledge of Git and configuration-as-code practices.\nIncident Management: Experience in incident triage, escalation, root cause analysis (RCA) and documentation.\nWillingness to work in a 24×7 rotational shift environment, including night shifts, weekends and public holidays.\n\nAdditional Requirements: \nStrong troubleshooting and analytical skills.\nAbility to work independently in a production support environment.\nGood communication, documentation, and incident coordination skills.\nComfortable working in a critical 24×7 production environment with strict SLA commitments.\nSkills\nGrafana, Linux, OTel, OpenTelemetry, Prometheus, RHEL, Ubuntu, VictoriaMetrics","description_format":"text","description_chars":5445,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Software Testing & QA","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-16T17:35:00Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":36,"reasons":["seen:0","win:early"],"computed_at":"2026-11-16T19:16:35Z"},"pay":null,"html_url":"https://alion.io/job/wits-innovation-lab-sre-engineer","json_url":"https://alion.io/job/wits-innovation-lab-sre-engineer.json","meta":{"generated_at":"2026-11-16T19:16:35Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3669,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":49824},"rest":"https://alion.io/mcp/rest/get_company?id=49824"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fwits-innovation-lab-sre-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fwits-innovation-lab-sre-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fwits-innovation-lab-sre-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/wits-innovation-lab-sre-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fwits-innovation-lab-sre-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4386345}},{"tool":"search_jobs","arguments":{"company":"thewitslab.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4386345},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}