Sr. Software Development Engineer

About Clearwater Analytics

Clearwater Analytics is a global SaaS leader in automated investment data aggregation, reconciliation, accounting, and reporting. Serving the world's largest asset managers, insurance companies, and corporations, Clearwater processes trillions of dollars in assets daily. Our Beacon Risk & Pricing Platform delivers real-time risk analytics and pricing intelligence - powered by reliable, high-quality market data and reference data pipelines at the core.

Role Overview

We are looking for a Senior Software Engineer to join the Beacon Risk & Pricing Platform team. In this role you will design, build, and operate mission-critical market data and reference data pipelines that feed our real-time pricing and risk engines. You will work closely with quant researchers, data vendors, and platform engineers to ensure that high-quality, low-latency data flows reliably across the entire investment analytics stack.

This is a high-impact, high-ownership role. You will make architectural decisions, mentor peers, and directly influence how Clearwater ingests, normalises, and distributes market data at scale.

What You'll Do

Market & Reference Data Pipelines

• Design and build scalable Python-based ETL/ELT pipelines ingesting multi-asset market data (equities, fixed income, derivatives, FX, commodities) from vendors such as Bloomberg, Refinitiv/LSEG, ICE Data Services, and FactSet.
• Engineer and maintain a robust securities master (reference data) system covering identifiers (CUSIP, ISIN, SEDOL, FIGI), corporate actions, instrument attributes, and classification hierarchies.
• Build real-time and batch data feeds using AWS-native services (Kinesis, MSK/Kafka, Glue, Lambda, Step Functions) with SLAs measured in milliseconds to seconds.
• Implement data quality frameworks - completeness checks, anomaly detection, stale-data alerting, and automated reconciliation against authoritative vendor sources.

Platform & Infrastructure

• Architect cloud-native, event-driven data infrastructure on AWS (S3, Redshift, RDS Aurora, DynamoDB, ElastiCache) with Infrastructure-as-Code (Terraform / CDK).
• Contribute to CI/CD pipelines and ensure deployments are automated, tested, and observable (CloudWatch, Datadog, OpenTelemetry).
• Drive schema design, data modelling, and partitioning strategies that balance query performance against storage cost at petabyte scale.

What You'll Bring

Required

• 7+ years of software engineering experience, with at least 4 years building production data pipelines in Python (Pandas, PySpark, asyncio, or equivalent).
• Deep hands-on experience with AWS data services: S3, Glue, Lambda, Kinesis / MSK, Redshift, RDS, Step Functions.
• Strong understanding of market data domains - pricing, corporate actions, identifier mapping, instrument reference data - from a financial services or fintech context.
• Experience designing and operating high-availability, low-latency streaming and batch pipelines in production.
• Proficiency in SQL and data modelling for analytical and operational workloads.
• Familiarity with data quality patterns: schema validation, reconciliation, SLA monitoring, dead-letter queues, and alerting.
• Strong communication skills - you write clear design docs, run productive tech discussions, and collaborate across disciplines.

Preferred

• Experience with securities master / reference data systems: CUSIP, ISIN, SEDOL, FIGI, and corporate action processing.
• Prior integration with major market data vendors (Bloomberg B-PIPE / SAPI, Refinitiv/LSEG Elektron, ICE Data Services).
• Knowledge of Apache Airflow or similar workflow orchestration tools.
• Experience with Apache Iceberg, Delta Lake, or other open table formats for data lake management.
• Familiarity with risk or pricing platforms in asset management or insurance investment operations.
• AWS Professional or Specialty certifications (Data Analytics, Solutions Architect).