{"id":327323,"url":"https://alion.io/job/indrive-data-engineer","title":"Senior Data Engineer","company":{"id":235,"name":"inDrive","domain":"indrive.com","url":"https://alion.io/company/indrive","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Pinpoint","truth_index":{"grade":"A","score":100,"open_postings":3,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":19,"computed_at":"2026-10-24T06:02:35Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Cairo, Egypt"],"countries":["EG"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"Airflow","optional":false},{"name":"Amazon S3","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AppsFlyer SDK","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"DLP","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Flyway","optional":false},{"name":"GCP","optional":false},{"name":"GitHub","optional":false},{"name":"Google BigQuery","optional":false},{"name":"IAM","optional":false},{"name":"Kubernetes","optional":false},{"name":"Least Privilege","optional":false},{"name":"Liquibase","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Python","optional":false},{"name":"Slack","optional":false},{"name":"SQL","optional":false},{"name":"Streamlit","optional":false},{"name":"Terraform","optional":false},{"name":"Flink","optional":true},{"name":"Java","optional":true}],"status":"live","first_seen_at":"2026-09-04T17:01:35Z","employer_posted_date":"2026-10-22","last_verified_at":"2026-10-25T01:07:57Z","board_verified":true,"closed_at":null,"days_open":50,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":50},"description":"We are looking for a Senior Data Engineer to join one of the Data Platform teams that works with the Marketing, Growth, partner, and financial data domains.\nYou will be working with cutting edge cloud technologies (GCP, AWS, BigQuery, Databricks, K8s) and building a large scale data infrastructure for analytics, machine learning, and streaming/CDC data delivery.\nKey Responsibilities\nBuild and operate batch and streaming ingestion into a layered BigQuery DWH (raw → ODS → data marts) using Airflow, Debezium CDC over Kafka with protobuf, Pub/Sub, and Dataflow\nIntegrate external data sources end-to-end - marketing platforms (GA4, AppsFlyer, TikTok/Meta/Google Ads), payment providers, S3 buckets, and third-party APIs - including schema contracts, backfills, and reconciliation\nEngineer the data platform itself in Python: custom Airflow operators and connectors in a shared ETL framework, Kafka Connect on Strimzi (K8s), Cloud Functions, and API integrations with external providers\nBuild CI/CD and change-management tooling for BigQuery: GitHub-based test-and-approval flows, SQL migration engines (Liquibase/Flyway/Bytebase), sandbox validation, backup and rollback\nOwn reliability and correctness of pipelines: idempotency, deduplication, late-data handling, backfill and replay, freshness monitoring and alerting; write integration and unit tests\nDrive data governance and compliance: ITGC-compliant change management for BigQuery, IAM and least-privilege access, PII policy tags and DLP, Unity Catalog on Databricks, column-level lineage (OpenMetadata/Dataplex), and disaster-recovery planning\nBuild internal data tools and platform services for agentic workflows with data - Streamlit apps, Slack bots, LLM-based agents and MCP servers that help teams find and use data\nSupport analysts and business teams with data requests, fostering data-driven decision-making across the company\nContribute to system design and architecture with the development team\nSkills, Knowledge and Expertise\nStrong practical Python: clean, well-structured, and tested code for services, tooling, and data pipelines\nSolid software design skills (OOP, modularity, design patterns) - we build platform tools for agentic workflows with data and plan to develop data-related backend services, so well-designed code is highly valued\nExperience building and operating services in a cloud environment (GCP, AWS or similar): CI/CD, containerization, monitoring and alerting\nFamiliarity with Kubernetes and Terraform - our infrastructure runs on GCP/K8s\nHands-on experience with DWH-related tasks (BigQuery or another cloud warehouse) and confident working SQL\nClear communication with non-engineering stakeholders - a meaningful share of the work is data requests from analysts and business teams\nDemonstrated ability to take ownership of technologies or services and proactively contribute ideas to the team\n Nice to have\nAdvanced SQL: complex queries, window functions, partitioning, clustering, and cost optimization\nExperience building reliable pipelines around CDC (e.g., Debezium): idempotency, schema evolution, backfills, and reconciliation\nAnalytical data modeling skills: table grain, facts vs dimensions, slowly changing dimensions, and metric definitions\nExperience with stream processing frameworks such as Flink or Apache Beam/Dataflow\nExposure to data governance and audit compliance (ITGC/SOX), Databricks Unity Catalog, or lineage/catalog tooling (OpenMetadata, Dataplex)\nInterest in building LLM-based agents and AI tooling for data\nWhy join us\nHelp us challenge injustice by creating fair choices for millions of people across 47 countries.\nDevelop your professional skills with access to mentoring, career consulting, and learning programs.\nCollaborate with teams around the world and gain international experience through our Global Talent Exchange Program.\nEngage in company-wide challenges, awards, sports activities, employee-led social impact and volunteering projects.\nWork alongside people who take initiative, speak openly, and challenge themselves to grow.\nImprove your language skills through co-financed courses and internal speaking clubs.\nFinal benefits may vary depending on the location.","description_format":"text","description_chars":4202,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Sports activities"],"hiring_locations":[{"name":"Egypt","iso":"EG","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Courier Services","Ride Hailing","Local Services Marketplaces"],"lifecycle":[{"event":"open","at":"2026-09-05T17:01:35Z"}],"visa":[],"liveness":{"score":17,"band":"cold","label":"Long shot","p_open":1,"p_active":0.609,"p_room":0.28,"age_days":49,"expected_fill_days":19,"reasons":["seen:1","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-24T06:02:35Z"},"pay":null,"html_url":"https://alion.io/job/indrive-data-engineer","json_url":"https://alion.io/job/indrive-data-engineer.json","meta":{"generated_at":"2026-10-25T04:33:45Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":6027,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-26T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":235},"rest":"https://alion.io/mcp/rest/get_company?id=235"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Findrive-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Findrive-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Findrive-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/indrive-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Findrive-data-engineer"}]}