Principal Kafka Platform Engineer

Experience: 8+ years (including 5+ years hands-on with Apache Kafka platform engineering)

Key Responsibilities

• Manage Kafka cluster lifecycle: provisioning, configuration tuning, upgrades, and decommissioning

• Deploy, operate, and maintain Kafka on Kubernetes using Confluent for Kubernetes (CFK)

• Monitor cluster health and set up alerting across consumer lag, broker health, and topic throughput

• Design and operate disaster recovery and multi-region replication strategies

• Enforce Kafka security policies including access reviews and ACL management

• Troubleshoot performance bottlenecks, consumer lag issues, and broker failures

Must Have

• 8+ years in messaging/streaming platform engineering, with 5+ years on Apache Kafka platform engineering

• Hands-on administration of self-managed on-premises Kafka clusters in KRaft mode

• Confluent for Kubernetes (CFK): operator management and custom resource configuration

• Kafka Connect: connector lifecycle, SMTs, and error handling

• Confluent Schema Registry; schema evolution and compatibility modes

• Kafka security: TLS/mTLS, SASL, and ACL configuration and troubleshooting

• Proficiency in Java or Scala; Python for scripting and tooling

• Observability: Prometheus, Grafana, and OpenTelemetry

Good to Have

• Apache Flink for complex event processing beyond Kafka Streams

• CDC pipelines into Kafka

• Operational experience with MirrorMaker 2 or Confluent Replicator in multi-DC setups

• Knowledge of event-driven architecture or data mesh principles

• Build and maintain infrastructure-as-code for Kafka resources

• Integrate Kafka platform changes with CI/CD pipelines for topic and schema deployments

Skills

Apache Kafka, Python, prometheus, grafana, Kubernetes, CI/CD, Java, Scala