Senior Software Engineer - Data Platform

We are looking for a hands-on Staff Software Engineer to join our Data Platform team. In this role, you will design and build the core data platform that powers our enterprise product. You'll work on building scalable data ingestion pipelines, real-time and batch processing systems, and a robust data lake architecture capable of handling large volumes of customer data. This is an opportunity to solve challenging distributed systems problems and build a modern cloud-native data platform from the ground up.

Responsibilities:

• Design and build scalable data ingestion frameworks to collect data from databases, SaaS applications, APIs, and event streams.
• Develop high-performance batch and real-time data pipelines capable of processing large-scale datasets.
• Build and evolve our cloud-native data lake/lakehouse architecture.
• Design reliable and fault-tolerant distributed systems for enterprise-scale workloads.
• Build reusable connectors for enterprise source systems such as Salesforce, HubSpot, PostgreSQL, MySQL, SQL Server, Oracle, MongoDB, and cloud storage platforms.
• Optimise data storage, partitioning, schema evolution, and query performance.
• Collaborate closely with product, AI/ML, and backend engineering teams to deliver scalable platform capabilities.
• Improve platform observability, monitoring, reliability, and operational excellence.
• Mentor engineers, drive engineering best practices, and influence technical direction across teams.

Requirements:

• 8-12 years of software engineering experience with strong expertise in distributed systems and data platforms.
• Strong programming skills in Java, Scala, Go, or Python.
• Experience building large-scale data ingestion and processing platforms.
• Hands-on experience with streaming technologies such as Apache Kafka, Apache Pulsar, or similar messaging systems.
• Experience building batch and streaming pipelines using Apache Spark, Apache Flink, Kafka Streams, or Apache Beam.
• Strong understanding of Change Data Capture (CDC) and data synchronisation techniques.
• Experience designing and operating modern data lake/lakehouse architectures using Apache Iceberg, Delta Lake, or Apache Hudi.
• Strong knowledge of distributed databases and storage systems.
• Experience with cloud platforms (AWS, Azure, or GCP) and Kubernetes-based deployments.
• Deep understanding of scalability, fault tolerance, distributed processing, and performance optimisation.
• Comfortable working in a fast-paced startup environment with a high degree of ownership.

Nice To Have:

• Experience with Debezium or other CDC frameworks.
• Experience with Trino, Presto, ClickHouse, Pinot, or Druid.
• Experience integrating enterprise applications such as Salesforce, SAP, HubSpot, Marketo, or Workday.
• Experience building multi-tenant SaaS platforms.
• Exposure to AI/ML data pipelines or feature stores.

Skills

beam, cdc, delta lake, flink, go, hudi, iceberg, java, kafka, kubernetes, pulsar, python, scala, spark