{"id":3483095,"url":"https://alion.io/job/kpler-senior-data-engineer-financial-flows","title":"Senior Data Engineer - Financial Flows","company":{"id":4764,"name":"Kpler","domain":"kpler.com","url":"https://alion.io/company/kpler","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"B","score":80,"open_postings":5,"ghost_share":0,"stale_share":0.8,"repost_share":0,"time_to_fill_p50_days":55,"computed_at":"2026-10-29T05:49:05Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":80000,"max_usd":157000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":52},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"ClickHouse","optional":false},{"name":"Docker","optional":false},{"name":"ElasticSearch","optional":false},{"name":"GitOps","optional":false},{"name":"Kubernetes","optional":false},{"name":"PHP","optional":false},{"name":"Python","optional":false},{"name":"QuestDB","optional":false},{"name":"Rest API","optional":false},{"name":"SQL","optional":false},{"name":"Time Series","optional":false},{"name":"WebSockets","optional":false}],"status":"live","first_seen_at":"2026-10-29T16:21:38Z","employer_posted_date":"2026-10-29","last_verified_at":"2026-10-30T01:10:13Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"About the Role\nAs a Senior Data Engineer on the Financial Flows team, you will take full technical ownership of the end-to-end data pipelines, predictive models, and cloud infrastructure powering Kpler’s commodity finance analytics. You will lead the modernization of our data foundation, spearheading the migration of production services to AWS and re-architecting legacy calculation engines into clean, performant Python services. Operating at the crossroads of quantitative research and cloud infrastructure, your work will directly ensure that market participants receive real-time, mission-critical insights across global commodities markets.\nKey Responsibilities\nDesign, build, and operate resilient batch and real-time streaming data ingestion pipelines connecting to external financial market data providers via APIs and WebSockets.\n\nLead the architectural migration of legacy backend infrastructure to modern cloud-native environments on AWS, establishing high standards for system reliability and scalability.\n\nDrive the refactoring and rewrite of the CTA positioning indicator calculation blocks from PHP to Python, ensuring strict mathematical output parity, comprehensive test coverage, and optimized latency.\n\nIngest and model complex financial datasets-including futures pricing, open interest, and trading volumes-across time-series stores and search engines like QuestDB and Elasticsearch.\n\nImplement automated data validation, proactive alerting, and system monitoring to ensure market data feeds remain accurate, continuous, and fault-tolerant.\n\nDeploy, maintain, and scale containerized services across Docker and Kubernetes environments using automated CI/CD pipelines and GitOps workflows.\n\nPartner closely with quantitative researchers, commodity analysts, and product managers to translate complex trading models into high-performance production pipelines.\n\nProvide technical mentorship to peer engineers, raising team-wide engineering rigor through thoughtful code reviews, system design guidance, and architectural documentation.\n\nExperience & Background\nWhat you'll need (Must-haves)\n5+ years of professional experience building, operating, and scaling production-grade data pipelines and backend infrastructure.\n\nAdvanced expertise in Python, with a strong background refactoring and rewriting production-critical code for performance and testability.\n\nProven hands-on experience building and maintaining real-time or streaming API ingestion pipelines (REST, WebSockets) handling time-series datasets.\n\nSolid production experience deploying and managing containerized applications on AWS using Docker, Kubernetes, and modern CI/CD practices.\n\nStrong database fundamentals in SQL and experience working with distributed search or time-series databases (e.g., Elasticsearch, QuestDB, ClickHouse).\n\nDeep experience establishing automated data quality frameworks, observability (monitoring/alerting), and root-cause troubleshooting for production systems.\n\nNice-to-haves\nDomain knowledge of financial market structures, futures contracts, open interest, and commodities trading symbology.\n\nExperience integrating data from major financial market vendors (e.g., Bloomberg, LSEG, Enverus, Trading Technologies).\n\nFamiliarity with legacy PHP codebase migration or reading PHP algorithms to translate into Python.\n\nPractical familiarity with GitOps frameworks and infrastructure-as-code tooling.","description_format":"text","description_chars":3415,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United Kingdom","iso":"GB","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Data Providers & Datasets","Supply Chain & Logistics Software","Financial Data & Market Intelligence"],"lifecycle":[{"event":"open","at":"2026-10-29T20:46:36Z"}],"visa":[{"country":"GB","licensed_sponsor":true,"evidence":"Licensed UK visa sponsor (Senior or Specialist Worker, Skilled Worker)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Global Business Mobility: Senior or Specialist Worker; Skilled Worker","cap_exempt":false,"checked_at":"2026-10-29T06:20:15+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":55,"reasons":["conf:2","win:early"],"computed_at":"2026-10-30T03:25:30Z"},"pay":null,"html_url":"https://alion.io/job/kpler-senior-data-engineer-financial-flows","json_url":"https://alion.io/job/kpler-senior-data-engineer-financial-flows.json","meta":{"generated_at":"2026-10-30T03:25:30Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4574,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4764},"rest":"https://alion.io/mcp/rest/get_company?id=4764"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fkpler-senior-data-engineer-financial-flows"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fkpler-senior-data-engineer-financial-flows"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fkpler-senior-data-engineer-financial-flows"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/kpler-senior-data-engineer-financial-flows\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fkpler-senior-data-engineer-financial-flows"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3483095}},{"tool":"search_jobs","arguments":{"company":"kpler.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3483095},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}