{"id":4437824,"url":"https://alion.io/job/phonepe-staff-engineer-kafka-codebase-and-internals","title":"Staff Engineer - Kafka Codebase and Internals","company":{"id":1554,"name":"PhonePe","domain":"phonepe.com","url":"https://alion.io/company/phonepe","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SmartRecruiters","truth_index":null},"role":"Backend","role_family":"Backend","seniority":"staff","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"Apache Pulsar","optional":false},{"name":"CockroachDB","optional":false},{"name":"etcd","optional":false},{"name":"ZooKeeper","optional":false}],"status":"live","first_seen_at":"2026-11-17T05:24:05Z","employer_posted_date":"2026-11-17","last_verified_at":"2026-11-17T06:30:12Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Our goal is to make digital payments so easy, safe and universally accepted that people never feel the need to carry cash or cards again. We believe India is at the cusp of a new mobile revolution, which will change the way we manage our money on the go. We see ourselves facilitating this change, through technology and dogged customer centricity.\nAt PhonePe, we take extra care to make sure you give your best at work, Everyday! And creating the right environment for you is just one of the things we do. We empower people and trust them to do the right thing. Here, you own your work from start to finish, right from day one.\nBeing enthusiastic about tech is a big part of being at PhonePe.\nIf you like building technology that impacts millions, ideating with some of the best minds in the country and executing on your dreams with purpose and speed, join us!\n About the Role\nWe are building an in-house team to maintain and contribute to our internal Apache Kafka distribution. This is not an operations role - you will work on the Kafka codebase itself: the broker, the KRaft controller, the protocol, the clients, and the storage layer. You will be our technical anchor for all things Kafka internals, driving upstream contributions, backporting critical fixes, and shaping our internal distribution to meet the needs of a large-scale production fleet.\nWe already do this successfully with GlusterFS, where we maintain an in-house distribution backed by deep upstream expertise. We want to replicate that model for Kafka.\nKey Responsibilities\nOwn our internal Kafka distribution: branch management, patch curation, release cutting, upgrade planning\nContribute upstream to Apache Kafka - bug fixes, KIPs, code reviews, design discussions\nDiagnose and fix deep broker/controller issues: replication, partition reassignment, KRaft consensus, log storage, ISR management, protocol handling\nBuild custom patches for production needs that aren't suitable for upstream (performance tuning for our workload profile, observability hooks, operational safety valves)\nEstablish engineering practices for Kafka development within the team: how we test, how we benchmark, how we validate patches before rolling them out\nMentor a supporting engineer and grow the team's capability over time\nAct as the internal authority on Kafka internals - advise platform teams on configuration, capacity planning, and architectural decisions that require knowledge of broker behavior\nAct as the primary interface for audit and compliance teams, ensuring PhonePe Kafka distribution meets various compliance and regulatory standards\n What are we looking for?\nDeep familiarity with the Kafka codebase - you have read, modified, and debugged the broker, controller, or client internals. Not just operated Kafka, but worked on it.\nTrack record of upstream contribution - committed patches, participated in KIP discussions, reviewed PRs, or equivalent engagement with the Apache Kafka community. Active committer/PMC status is strongly preferred but not strictly required if your contribution history demonstrates equivalent depth.\nStrong distributed systems fundamentals - consensus protocols, replication, exactly-once semantics, log-structured storage, distributed state machines. You understand these not as buzzwords but as engineering tradeoffs you've navigated.\nSystems-level programming comfort - JVM internals (GC tuning, memory layout, JIT behavior), OS-level I/O (page cache, fsync semantics, zero-copy), networking (TCP tuning, socket buffers). Kafka performance work lives at this layer.\nExperience maintaining a fork or distribution - you've managed the tension between carrying local patches and staying mergeable with upstream. You know what it costs to diverge and how to minimize that cost. • \n8+ years of experience in systems/infrastructure software engineering, with a meaningful portion spent on Kafka or comparable distributed systems (Pulsar, Bookkeeper, Zookeeper, etcd, CockroachDB, etc.)\n Good to have\nApache Kafka Committer or PMC member\nExperience with KRaft (the ZooKeeper-free controller)\nAuthored or co-authored a KIP\nExperience with Kafka Streams or Kafka Connect internals (not just usage)`\nFamiliarity with Kafka's test infrastructure (system tests, Trogdor, ducktape)","description_format":"text","description_chars":4275,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"English","level":"All levels","optional":false}]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Payment Processing & Gateways","Money Transfer & Remittances","Investing Apps & Robo-Advisors"],"lifecycle":[{"event":"open","at":"2026-11-17T06:30:12Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":28,"reasons":["conf:1","win:early"],"computed_at":"2026-11-17T07:53:59Z"},"pay":null,"html_url":"https://alion.io/job/phonepe-staff-engineer-kafka-codebase-and-internals","json_url":"https://alion.io/job/phonepe-staff-engineer-kafka-codebase-and-internals.json","meta":{"generated_at":"2026-11-17T07:53:59Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":14183,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1554},"rest":"https://alion.io/mcp/rest/get_company?id=1554"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fphonepe-staff-engineer-kafka-codebase-and-internals"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fphonepe-staff-engineer-kafka-codebase-and-internals"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fphonepe-staff-engineer-kafka-codebase-and-internals"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/phonepe-staff-engineer-kafka-codebase-and-internals\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fphonepe-staff-engineer-kafka-codebase-and-internals"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4437824}},{"tool":"search_jobs","arguments":{"company":"phonepe.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4437824},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}