Kafka Administrator

Responsibilities:

• Install, configure, and maintain Apache Kafka clusters in production and non-production environments.
• Monitor Kafka brokers, topics, partitions, consumer groups, and cluster health.
• Perform capacity planning, performance tuning, and resource optimisation.
• Manage topic creation, retention policies, replication, and partition strategies.
• Troubleshoot Kafka-related issues involving producers, consumers, brokers, and Zookeeper/KRaft.
• Implement and manage Kafka security features, including SSL, SASL, ACLs, and encryption.
• Perform cluster upgrades, patching, migration, and disaster recovery activities.
• Configure and maintain Kafka Connect, Schema Registry, MirrorMaker, and Confluent components.
• Develop automation scripts for Kafka administration and operational tasks.
• Ensure high availability, scalability, and fault tolerance of Kafka platforms.
• Work closely with application, DevOps, infrastructure, and cloud teams.
• Create and maintain operational documentation, SOPs, and monitoring dashboards.
• Provide production support and participate in on-call rotations.

Requirements:

• Strong hands-on experience with Apache Kafka administration.
• Knowledge of Kafka architecture, brokers, topics, partitions, replication, and consumer groups.
• Experience with Kafka Connect, Schema Registry, and Kafka Streams.
• Strong understanding of Linux/Unix administration.
• Experience with monitoring tools such as Prometheus, Grafana, ELK, Splunk, or Datadog.
• Knowledge of Kafka security (SSL, SASL, Kerberos, ACLs).
• Experience in troubleshooting performance and connectivity issues.
• Familiarity with scripting languages such as Shell, Python, or Bash.
• Understanding of distributed systems and messaging architectures.

Preferred Skills:

• Experience with Confluent Platform administration.
• Knowledge of Kubernetes, Docker, and containerised Kafka deployments.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Exposure to CI/CD tools, including Jenkins, GitHub Actions, or Azure DevOps.
• Knowledge of Event-Driven Architecture (EDA) and microservices.

Skills

Apache Kafka, Confluent, Kafka, Kafka Connect, Kafka Streams, Linux, Schema Registry, Unix