Senior Streaming Data Developer

Role Summary :

The Senior Streaming Data Developer / Confluent Lead is responsible for designing, developing, governing, and supporting enterprise-scale event streaming and real-time data processing solutions using the Confluent Platform and Apache Kafka ecosystem.

This role combines deep hands-on engineering expertise with architectural leadership to deliver scalable, secure, resilient, and high-performing streaming data solutions.

The candidate will work closely with enterprise architects, platform teams, business stakeholders, and delivery partners to drive adoption of event-driven and real-time streaming architectures across the organization.

The ideal candidate should possess strong practical experience with Confluent Managed Services, Apache Flink, SQL-based stream processing, and Kafka ecosystem tooling, while also being capable of guiding enterprise-level streaming strategy and implementation standards.

Experience Required :

812 years of overall IT experience with at least 6+ years of hands-on experience in :

- Confluent Platform / Apache Kafka

- Real-time event streaming solutions

- Streaming data engineering and processing

- Strong hands-on experience with Confluent Cloud / Confluent Managed Services

- Proven experience designing and implementing enterprise-scale event-driven architectures

- Strong practical experience with Apache Flink for stateful stream processing, event enrichment, event generation, windowing, and real-time analytics

- Strong SQL expertise (mandatory), particularly for Flink SQL and streaming transformations

- Experience migrating or modernizing streaming solutions from KSQL/ksqlDB to Flink-based implementations

- Experience working within large, complex enterprise integration and data ecosystems

- Experience working with managed connectors and custom connector development

Key Responsibilities :

1. Streaming Architecture & Platform Leadership :

- Define and govern target-state architecture for Kafka and Confluent-based event streaming platforms

- Provide technical leadership for enterprise streaming initiatives and real-time data integration programs

- Validate solution direction and provide strategic guidance to business and technology stakeholders

- Support client teams in defining scalable and maintainable streaming data patterns

2. Hands-on Development & Engineering :

- Develop and support Kafka producers, consumers, Kafka Connect integrations, and Flink streaming applications

- Design and implement real-time data transformation and enrichment pipelines using Flink and SQL

- Build and customize Kafka connectors where managed connectors do not meet business requirements

- Provide hands-on troubleshooting, performance tuning, and optimization for Kafka and Flink workloads

- Support migration and transformation approaches from KSQL/ksqlDB to Apache Flink

3. Data Governance & Streaming Standards :

- Establish and govern topic design, naming conventions, schema governance, event lifecycle, and data contracts

- Define and manage append vs upsert topic strategies, Avro/JSON schema registry, and compatibility rules

- Ensure streaming data solutions align with enterprise architecture, security, and governance standards

4. Platform Reliability & Operations :

- Provide guidance on high availability, disaster recovery, scalability, and resiliency

- Support Confluent Managed Services operations and platform governance

- Collaborate with cloud, DevOps, data engineering, and integration teams for consistent platform adoption

Required Skills & Technical Expertise :

Core Streaming Technologies :

- Confluent Platform / Confluent Cloud, Apache Kafka, Apache Flink, Kafka Connect, Schema Registry, REST Proxy, ksqlDB

Kafka & Streaming Concepts :

- Producers/Consumers, Partitions/Replication, Consumer Groups, Offset Management, Topic Compaction, Exactly-once processing, Event-driven architecture

Flink & Stream Processing :

- Flink SQL, Stateful stream processing, Stream joins, Windowing, Event-time processing

Data & Schema Management :

- SQL expertise, Avro/JSON schemas, Schema evolution, CDC patterns

Cloud & Platform Engineering :

- Kafka in cloud/Kubernetes, CI/CD, Infrastructure-as-Code, performance tuning

Soft Skills :

- Strong stakeholder communication, documentation, and governance capabilities

Preferred Experience :

- Multi-region Kafka deployments, enterprise data/AI platform integration, regulated environments, CDC, certifications

Skills

Confluence, Confluent Kafka, Data Engineering, Streaming API, Streaming Protocols, Apache Flink, SQL, Data Governance, KQL