{"id":4051790,"url":"https://alion.io/job/opinov8-staff-data-engineer-databricks-specialist","title":"Staff Data Engineer (Databricks specialist)","company":{"id":2127539,"name":"Opinov8","domain":"opinov8.com","url":"https://alion.io/company/opinov8","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Teamtailor","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"staff","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_regions","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["DE","FR","GB","AT","BE","BG","HR","CZ","DK","FI","HU","IE","IT","NL","NO","PL","RO","ES","SE","CH","EE","GR","IS","LV","LI","LT","LU","MT","PT","SK","SI"],"hiring_countries_total":31,"salary":null,"salary_estimate":null,"experience_years_min":12,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"AutoGen","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Data Vault","optional":false},{"name":"Databricks","optional":false},{"name":"Embeddings","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Feature Store","optional":false},{"name":"GCP","optional":false},{"name":"Google BigQuery","optional":false},{"name":"GraphQL","optional":false},{"name":"Informatica","optional":false},{"name":"Java","optional":false},{"name":"LangChain","optional":false},{"name":"LLM","optional":false},{"name":"LLMOps","optional":false},{"name":"Machine Learning","optional":false},{"name":"Master Data Management","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Rest API","optional":false},{"name":"Scala","optional":false},{"name":"Semantic Kernel","optional":false},{"name":"Snowflake","optional":false},{"name":"SQL","optional":false},{"name":"Alation","optional":true},{"name":"Amazon Kinesis","optional":true},{"name":"Apache Kafka","optional":true},{"name":"Collibra","optional":true},{"name":"Dagster","optional":true},{"name":"dbt","optional":true},{"name":"Docker","optional":true},{"name":"Flink","optional":true},{"name":"Kubernetes","optional":true},{"name":"Multi-Agent Systems","optional":true},{"name":"Spark","optional":true}],"status":"live","first_seen_at":"2026-11-10T20:58:46Z","employer_posted_date":"2026-11-10","last_verified_at":"2026-11-11T02:07:22Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"PROJECT OVERVIEW\nOur client is a leading commercial real estate advisory and services provider, supporting large institutional investors, global corporations, and other owners and occupiers. Their mission is to make leasing, buying, and selling properties across the retail, office, multifamily, industrial, student housing, and hospitality sectors easier and more efficient.\nIN THIS ROLE, YOU WILL\nOwn and drive the technical architecture for complex, cross-team data initiatives spanning ingestion, transformation, storage, and serving layers.\n\nDesign, build, and maintain scalable, high-performance data pipelines and distributed data platforms in a cloud-native environment (Azure, AWS, or GCP).\n\nArchitect and lead enterprise Master Data Management (MDM), including golden records, entity resolution, data domains, reference and hierarchy management, and stewardship, to create trusted, authoritative data across the business.\n\nArchitect and integrate agentic AI and LLM-driven workflows (autonomous agents, RAG pipelines, AI copilots) into data platforms and pipelines to drive efficiency and new capabilities.\n\nBuild and support the data foundations for machine learning and AI, including feature stores, vector stores, embeddings, and ML/LLMOps pipelines.\n\nDesign and deliver backend data services and APIs (REST/GraphQL), and contribute across the stack to expose curated datasets to applications, analytics, and BI consumers.\n\nSet engineering standards and best practices for data quality, modeling, testing, observability, and deployment across the organization, including responsible use of AI-assisted development tools.\n\nEstablish data governance, lineage, cataloging, and quality frameworks across the data estate.\n\nLead technical design reviews and provide architectural guidance to multiple engineering and data teams.\n\nPartner with product, analytics, and engineering leadership to translate business strategy into scalable data roadmaps, including AI-driven capabilities.\n\nIdentify and resolve systemic performance, reliability, and scalability issues across the data stack.\n\nMentor and coach senior and mid-level engineers, raising the technical bar across the organization on data engineering, MDM, and AI practices.\n\nDrive adoption of modern frameworks, tools, and engineering practices, including agentic AI and LLM tooling, to improve delivery velocity and platform resilience.\n\nMaintain awareness of emerging technologies and industry trends, particularly in agentic AI, master data management, and modern data platforms, and assess their applicability to the business.\n\nIF YOU ARE\nBachelor's degree in Computer Science, Engineering, MIS, or related field preferred.\n\n12+ years of experience in data engineering or software engineering, with demonstrated experience architecting data platforms and pipelines at scale.\n\nExpert-level SQL and strong proficiency in Python (Scala or Java a plus) for large-scale data processing and transformation.\n\nDeep experience with cloud data platforms (e.g., Databricks, Snowflake, Synapse, BigQuery, Redshift) and cloud-native architecture patterns.\n\nDeep understanding of distributed systems, data modeling (dimensional, data vault, lakehouse), and ETL/ELT architecture.\n\nHands-on experience designing and implementing Master Data Management (MDM) solutions, including entity resolution, match/merge, golden records, and reference/hierarchy management (e.g., Informatica, Reltio, Profisee, or similar).\n\nHands-on experience building or integrating agentic AI systems, LLM-powered applications, RAG pipelines, or AI agent orchestration frameworks (e.g., LangChain, AutoGen, Semantic Kernel, MCP).\n\nExperience building backend data services and APIs (REST/GraphQL), with comfort working across the full stack.\n\nStrong background with both relational (SQL) and NoSQL data stores, plus data lake/lakehouse formats (Delta, Iceberg, Parquet).\n\nDeep understanding of CI/CD pipelines, infrastructure as code, and DevOps/DataOps practices.\n\nProven track record of leading large-scale technical initiatives across multiple teams.\n\nDemonstrated ability to mentor engineers and influence technical direction without direct reporting authority.\n\nNICE TO HAVE\nExperience with data governance, lineage, and cataloging tools (e.g., Unity Catalog, Microsoft Purview, Collibra, Alation).\n\nExperience designing multi-agent systems, tool-calling architectures, or retrieval-augmented generation (RAG) pipelines.\n\nExperience with event-driven architectures and streaming/real-time data processing (e.g., Kafka, Event Hubs, Kinesis, Flink, Spark Structured Streaming).\n\nExperience building the data layer for ML/AI, including feature stores, vector databases, embeddings, and ML/LLMOps.\n\nFamiliarity with containerization and orchestration (Docker, Kubernetes) and workflow orchestration (Airflow, Dagster, dbt).\n\nPrior experience in commercial real estate, fintech, or operations/transaction systems.\n\nTrack record of speaking, writing, or open-source contributions that demonstrate technical thought leadership, especially in applied AI or data.\n\nAS AN OPINOV8R, YOU WILL HAVE\nDigital-First Approach: Great talent knows no borders! You can work from wherever you are - we hire and collaborate with professionals worldwide.\n\nRemote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere.\n\nExciting Projects: Dive into impactful projects across industries that challenge and spark creativity.\n\nBoost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience.\n\nJoin the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.\n\nHOW’S THE HIRING PROCESS GOING\nWe strive to make our hiring process smooth and transparent to find the perfect match for both sides. Steps may differ depending on the role, but here’s what to expect:\nInitial Interview: If your background fits the role, we’ll invite you for an interview with a Talent Acquisition Specialist.\n\nTechnical Interview: Depending on the position, you may complete a technical assessment or test task.\n\nClient Interview: You can expect 1 or 2 Client Interviews.\n\nFinal Decision: After all steps, we’ll get back to you with the result and next steps.","description_format":"text","description_chars":6346,"description_truncated":false,"requirements":{"experience_years_min":12,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Continuous learning","Flexible schedule"],"hiring_locations":[{"name":"Germany","iso":"DE","kind":"region"},{"name":"France","iso":"FR","kind":"region"},{"name":"United Kingdom","iso":"GB","kind":"region"},{"name":"Austria","iso":"AT","kind":"region"},{"name":"Belgium","iso":"BE","kind":"region"},{"name":"Bulgaria","iso":"BG","kind":"region"},{"name":"Croatia","iso":"HR","kind":"region"},{"name":"Czech Republic","iso":"CZ","kind":"region"},{"name":"Denmark","iso":"DK","kind":"region"},{"name":"Finland","iso":"FI","kind":"region"},{"name":"Hungary","iso":"HU","kind":"region"},{"name":"Ireland","iso":"IE","kind":"region"},{"name":"Italy","iso":"IT","kind":"region"},{"name":"Netherlands","iso":"NL","kind":"region"},{"name":"Norway","iso":"NO","kind":"region"},{"name":"Poland","iso":"PL","kind":"region"},{"name":"Romania","iso":"RO","kind":"region"},{"name":"Spain","iso":"ES","kind":"region"},{"name":"Sweden","iso":"SE","kind":"region"},{"name":"Switzerland","iso":"CH","kind":"region"},{"name":"Estonia","iso":"EE","kind":"region"},{"name":"Greece","iso":"GR","kind":"region"},{"name":"Iceland","iso":"IS","kind":"region"},{"name":"Latvia","iso":"LV","kind":"region"},{"name":"Liechtenstein","iso":"LI","kind":"region"},{"name":"Lithuania","iso":"LT","kind":"region"},{"name":"Luxembourg","iso":"LU","kind":"region"},{"name":"Malta","iso":"MT","kind":"region"},{"name":"Portugal","iso":"PT","kind":"region"},{"name":"Slovakia","iso":"SK","kind":"region"},{"name":"Slovenia","iso":"SI","kind":"region"}],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-11T02:07:22Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":29,"reasons":["conf:2","win:early"],"computed_at":"2026-11-11T04:15:08Z"},"pay":null,"html_url":"https://alion.io/job/opinov8-staff-data-engineer-databricks-specialist","json_url":"https://alion.io/job/opinov8-staff-data-engineer-databricks-specialist.json","meta":{"generated_at":"2026-11-11T04:15:08Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4419,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-12T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2127539},"rest":"https://alion.io/mcp/rest/get_company?id=2127539"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fopinov8-staff-data-engineer-databricks-specialist"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fopinov8-staff-data-engineer-databricks-specialist"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fopinov8-staff-data-engineer-databricks-specialist"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/opinov8-staff-data-engineer-databricks-specialist\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fopinov8-staff-data-engineer-databricks-specialist"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4051790}},{"tool":"search_jobs","arguments":{"company":"opinov8.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4051790},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}