Senior Data Engineer
SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.
Responsibilities:
Design and implement streaming solutions using Apache Flink and Apache Kafka.
Develop Flink SQL-based capabilities including filtering, payload projections, and topic joins.
Deploy, configure, and operate Flink workloads using Flink Kubernetes Operator.
Build and maintain Kafka Connect integrations and transformation logic.
Develop XML-to-Avro transformation capabilities and custom Kafka Connect SMT plugins.
Collaborate with internal teams on schema governance and self-service deployment frameworks.
Support CI/CD and GitOps deployment processes.
Deliver solutions using AI-assisted engineering practices throughout the software development lifecycle.
Optimize platform performance, scalability, and operational efficiency.
Contribute to stream-processing architecture and engineering best practices.
Skills:
Strong hands-on experience with Apache Flink.
Expertise in Flink SQL including filtering, payload projections, and joins.
Experience with Flink Kubernetes Operator and stateful stream processing.
Strong knowledge of Apache Kafka and Confluent Platform.
Experience with Kafka Connect administration and connector development.
Experience developing and managing SMT (Single Message Transform) plugins.
Knowledge of Confluent Schema Registry and schema governance concepts.
Experience with Avro schema design and schema evolution.
Hands-on experience deploying and operating workloads on Kubernetes.
Knowledge of CI/CD and GitOps deployment approaches.
Proven experience using AI-assisted development tools across design, development, testing, documentation, and delivery.
Nice to have:
Experience with Flink Table API.
Experience designing self-service platform capabilities.
Background in large-scale event-driven architectures.
Follow us on Linkedin! http://linkedin.com/company/snisourcing/