{"id":1984599,"url":"https://alion.io/job/callista-group-senior-data-engineer-azure-databricks","title":"Senior Data Engineer (Azure Databricks)","company":{"id":3906069,"name":"Callista Group","domain":"callistagroup.com","url":"https://alion.io/company/callista-group","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"contractor","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Frankfurt am Main, Germany"],"countries":["DE"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":78000,"max_usd":140000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":22},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"CI/CD","optional":false},{"name":"Data Vault","optional":false},{"name":"Databricks","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Python","optional":true},{"name":"Rest API","optional":true}],"status":"live","first_seen_at":"2026-10-06T20:30:23Z","employer_posted_date":null,"last_verified_at":"2026-10-06T20:30:23Z","board_verified":false,"closed_at":null,"days_open":6,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":6},"description":"Für unseren Kunden, ein Versicherungsunternehmen im Großraum Frankfurt am Main, suchen wir im Rahmen des Aufbaus einer neuen dispositiven Datenplattform (Lakehouse) auf Basis von Microsoft Azure und Databricks zwei erfahrene Data Engineers. Du baust die Datenstrecken von den Quellsystemen bis in das Core-Modell und übernimmst dabei die technische Federführung in der Umsetzung. Du arbeitest im Tandem mit Mitarbeitenden des Kunden und gibst dein Databricks-Know-how aktiv weiter.\n\nAufgaben\nAnbindung von Quellsystemen und externen Dienstleistern (Delta-Lieferungen, Dateien, APIs) sowie eines CRM-Systems inklusive automatisierter Ladeprozesse\n\nEntwicklung von ETL/ELT-Strecken mit Delta-Verarbeitung, Fehlerbehandlung und nachvollziehbaren Wiederanlaufverfahren\n\nUmsetzung der Dublettenbereinigung und Zusammenführung von Kunden- und Vermittlerdaten\n\nImplementierung des Core-Datenmodells mit Historisierung entlang der Medallion-Architektur (Databricks, PySpark, SQL)\n\nUmsetzung von Datenqualitätsregeln, Prüfroutinen, Monitoring und Alerting\n\nBereitstellung von Exporten und Rückflüssen an operative Systeme (z. B. Kampagnenselektionen für das CRM)\n\nMitarbeit an Entwicklungspipeline und CI/CD, Rollen und Berechtigungen, Data Catalog und Lineage\n\nDokumentation, Runbooks und Wissenstransfer im Tandem mit dem Kundenteam\n\n Anforderungen\nMehrjährige Erfahrung als Data Engineer, idealerweise im Aufbau von Data-Warehouse- oder Lakehouse-Lösungen\n\nSehr gute Kenntnisse in Databricks, PySpark/Spark, SQL und Delta Lake\n\nPraxis mit Databricks Workflows bzw. Azure Data Factory und weiteren Azure-Datenservices\n\nErfahrung in Datenmodellierung und Historisierung (Data Vault und/oder dimensionale Modellierung)\n\nKenntnisse in Git, CI/CD und automatisiertem Testing von Datenpipelines\n\nErfahrung mit KI-gestützter Entwicklung (z. B. Coding-Assistenten, agentische Workflows) von Vorteil\n\nErfahrung mit Quellsystemen und Daten im Versicherungsumfeld von Vorteil\n\nFreude an Wissensvermittlung und Zusammenarbeit im gemischten Team\n\nDatabricks- oder Azure-Data-Engineer-Zertifizierung von Vorteil\n\nVerhandlungssichere Deutsch- und gute Englischkenntnisse\n\nGenerelle Rahmenbedingungen\nPensum: 100 %\n\nStart: Anfang November 2026\n\nDauer: bis 31.03.2027 mit Option auf Verlängerung (Folgephasen geplant bis Ende 2028)\n\nArbeitsort: Großraum Frankfurt am Main (Deutschland)\n\nOnsite/Remote: In den ersten 2–3 Monaten hoher Onsite-Anteil (3–4 Tage pro Woche), danach ca. 2 Tage onsite, Rest remote\n\nKundengespräch: Verfügbarkeit für Interviews beim Kunden Ende Oktober 2026\n\nFind Jobs in Germany on Arbeitnow","description_format":"text","description_chars":2595,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Germany","iso":"DE","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-07T01:33:00Z"}],"visa":[],"liveness":{"score":89,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.894,"p_room":1,"age_days":6,"expected_fill_days":22,"reasons":["seen:6","velocity","win:early"],"computed_at":"2026-10-13T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/callista-group-senior-data-engineer-azure-databricks","json_url":"https://alion.io/job/callista-group-senior-data-engineer-azure-databricks.json","meta":{"generated_at":"2026-10-13T18:24:05Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":28148,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-14T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3906069},"rest":"https://alion.io/mcp/rest/get_company?id=3906069"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcallista-group-senior-data-engineer-azure-databricks"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcallista-group-senior-data-engineer-azure-databricks"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcallista-group-senior-data-engineer-azure-databricks"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/callista-group-senior-data-engineer-azure-databricks\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcallista-group-senior-data-engineer-azure-databricks"}]}