{"id":3456314,"url":"https://alion.io/job/capgemini-senior-data-engineer-pythonspark-ms-fabric","title":"Senior Data Engineer (Python/Spark/ MS Fabric)","company":{"id":145,"name":"Capgemini","domain":"capgemini.com","url":"https://alion.io/company/capgemini","size_band":"5000+","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Career site","truth_index":{"grade":"B","score":79,"open_postings":1757,"ghost_share":0.005,"stale_share":0.816,"repost_share":0.018,"time_to_fill_p50_days":30,"computed_at":"2026-11-09T06:03:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Ukraine"],"countries":["UA"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":19500,"max_usd":52000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":2679},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Microsoft Fabric","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false}],"status":"closed","first_seen_at":"2026-10-15T12:44:12Z","employer_posted_date":"2026-10-29","last_verified_at":"2026-11-08T19:09:34Z","board_verified":false,"closed_at":"2026-11-08T19:09:34Z","days_open":24,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":24},"description":"Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.\nJob Description\nTime Zone Requirement\nResources must be able to provide overlap with the customer's US Eastern Time working hours, including availability through approximately 4:30 PM ET.\nAbout Client\nOur client is expanding its data platform team as part of its ongoing data modernization initiatives. They are looking to add two hands-on Data Engineers to support the evolution of the Microsoft-based analytics and data platform.\nRole Overview\nDesign, optimize, and operate reliable, large-scale, cost-aware Spark and Delta Lake workloads.\nBuild and optimize Spark pipelines on Microsoft Fabric or Databricks. Diagnose performance using Spark UI and Delta statistics. Manage Delta Lake design, Fabric capacity consumption, compute sizing, throttling exposure, and operational reliability. Create reusable Python libraries and Spark modules for ingestion, governance, security, monitoring, and optimization.\nDemonstrated production troubleshooting and capacity-economics experience, not only notebook or ETL development.\nMeasures workload behavior before changing infrastructure; optimizes code and configuration before increasing compute; documents pool sizing, throttling risk, and operational requirements; explains capacity behavior to platform and engineering teams.\nMust Have\n· Microsoft Fabric or Databricks production engineering\n· Apache Spark and Spark UI troubleshooting\n· Shuffle skew, spill, OOM, task-duration analysis, AQE, join strategy, and predicate pushdown\n· Delta Lake optimization: liquid clustering, partitioning, Z-order, file sizing, OPTIMIZE, statistics, VACUUM, and incremental MERGE\n· Fabric capacity economics: CU-seconds, bursting, smoothing, throttling stages, pool configuration, node sizing, concurrency, and workload isolation\n· Production Python library development, packaging, validation, logging, testing, versioning, and documentation\n· Operational pipeline design: idempotency, watermarks, quarantine, monitoring, failure recovery, and SLA awareness\nGood-To-Have Skills\n· OneLake security and access roles\n· Governance metadata and TBLPROPERTIES\n· Reusable ingestion and maintenance frameworks\n· Secrets handling and thread safety\n· Direct Lake optimization awareness\nWhat You Will Love About Working Here\nWe care about all our employees and want them to feel as comfortable as possible. That's why we offer them health insurance from the first days, regardless of the probationary period.\nThe gift from the company - Christmas holidays from 25 December to 31 December.\nСooperation with Superhumans center and Veteran HUB. Capgemini Engineering has supported the launch of psychological rehabilitation department of Superhumans. Our team also donated over UAH 500 000 prosthetics for three Ukrainian defenders. Currently, we support psychological counseling provided by the Veteran Hub, and we have implemented an internal policy making the company friendly to military and veterans with the assistance of the Hub.\n\nCapgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of €22.1 billion.\nMake it real | www.capgemini.com","description_format":"text","description_chars":4002,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Health insurance"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Systems Integrators","Management Consulting","Engineering Services"],"lifecycle":[{"event":"open","at":"2026-10-29T12:32:27Z"},{"event":"close","at":"2026-11-08T19:09:34Z"}],"visa":[],"liveness":null,"pay":null,"html_url":"https://alion.io/job/capgemini-senior-data-engineer-pythonspark-ms-fabric","json_url":"https://alion.io/job/capgemini-senior-data-engineer-pythonspark-ms-fabric.json","meta":{"generated_at":"2026-11-09T10:01:00Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":10203,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-10T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":145},"rest":"https://alion.io/mcp/rest/get_company?id=145"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcapgemini-senior-data-engineer-pythonspark-ms-fabric"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcapgemini-senior-data-engineer-pythonspark-ms-fabric"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcapgemini-senior-data-engineer-pythonspark-ms-fabric"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/capgemini-senior-data-engineer-pythonspark-ms-fabric\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcapgemini-senior-data-engineer-pythonspark-ms-fabric"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3456314}},{"tool":"search_jobs","arguments":{"company":"capgemini.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3456314},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}