{"id":3805616,"url":"https://alion.io/job/imizizi-data-engineer-expert","title":"Data Engineer (Expert)","company":{"id":4496229,"name":"Imizizi","domain":"imizizi.co.za","url":"https://alion.io/company/imizizi","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pretoria, South Africa"],"countries":["ZA"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Angular","optional":false},{"name":"Apache Kafka","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Grafana","optional":false},{"name":"Hadoop","optional":false},{"name":"Java","optional":false},{"name":"Kubernetes","optional":false},{"name":"Least Privilege","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"Terraform","optional":false},{"name":"Zero Trust","optional":false},{"name":"JavaScript","optional":true},{"name":"TypeScript","optional":true}],"status":"live","first_seen_at":"2026-11-05T13:01:05Z","employer_posted_date":null,"last_verified_at":"2026-11-05T13:01:05Z","board_verified":false,"closed_at":null,"days_open":2,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":2},"description":"Reference: JHB001567-KF-2\n\nESSENTIAL SKILLS:\n\nHands-on experience with Kafka and event streaming platforms for real-time data movement.\n\nProven experience with API integration patterns, webhooks and event/webhook ingestion.\n\nStrong proficiency in Python for data engineering, ingestion pipelines and automation.\n\nSolid competence with enterprise databases and query languages, with practical experience with database performance tuning and query optimisation for OLTP/operational workloads.\n\nExperience in data modelling to design schemas and standardized data representations.\n\nExperience with schema registries and contract-first designs (Avro, Protobuf) to manage producer/consumer compatibility.\n\nStrong understanding and practice of data quality techniques and tooling to ensure trusted data.\n\nKnowledge of metadata management and cataloguing to support discoverability and lineage.\n\nFamiliarity with ETL/ELT patterns and best practices for performant, reliable data pipelines.\n\nAbility to normalise and correlate data across multiple sources (CMDB, identity, security tooling).\n\nDesign and operate idempotent pipelines with retry, replay and compensation mechanisms.\n\nADVANTAGEOUS SKILLS:\n\nExperience with Java for stream processing or connector development.\n\nAwareness of frontend frameworks (e.g., Angular) to better understand downstream consumers.\n\nFamiliarity with infrastructure automation and IaC (e.g., Terraform) to deploy integration components.\n\nExperience operating container platforms and orchestration (Kubernetes) for scalable stream processing.\n\nExperience with NoSQL/document stores such as MongoDB for operational data needs.\n\nFamiliarity with enterprise systems like SAP and working with their integration interfaces.\n\nExperience with big data ecosystems (CDH/Hadoop) and distributed storage/processing.\n\nWorking knowledge of Azure Synapse or similar data platform services for integrated data processing.\n\nObservability for streaming: experience with metrics, tracing and logging for pipelines (Prometheus, Grafana, OpenTelemetry).\n\nExperience with connector ecosystems and managed services (Kafka Connect, Confluent Cloud, managed Kafka).\n\nKnowledge of message delivery semantics, partitioning strategies and capacity planning for high-throughput pipelines.\n\nDuties & Responsibilities\nROLE & RESPONSIBILITIES:\n\nDesign, build and operate operational data integration solutions focused on making enterprise data available, trusted and consumable.\n\nIngest and stream data from Zero Trust data sources (CMDB, identity systems, security tooling) into operational pipelines rather than analytics stores.\n\nBuild and maintain Kafka topics, producers/consumers, connectors and streaming applications to enable realtime flows.\n\nImplement API-based integrations, webhook listeners and file ingestion solutions to capture operational events.\n\nDevelop robust ETL/ELT and stream processing logic to normalise, correlate and enrich data across sources.\n\nEnsure pipelines are idempotent and resilient (retries, replay, backpressure and compensation strategies).\n\nApply data modelling and metadata practices to ensure consistent schemas and discoverability.\n\nImplement and monitor data quality checks, anomaly detection and validation rules; drive remediation where needed.\n\nMaintain data lineage, access control and security practices to meet governance and compliance requirements.\n\nCollaborate with automation and orchestration teams to ensure streamed operational data feeds automation workflows and runbooks.\n\nOwn pipeline alerting, runbooks, and participate in incident response/on-call rotations\n\nTroubleshoot production issues, perform root cause analysis and implement long-term reliability improvements.\n\nBuild testable pipelines with unit/integration tests, contract tests and CI/CD deployment pipelines for streaming workloads\n\nData Quality Gates: enforce automated quality checks early in pipelines (validation, schema checks, anomaly detection) and block or quarantine bad data.\n\nPerformance & Cost Awareness: implement efficient partitioning, batching and resource usage patterns while tracking cost and throughput trade-offs.\n\nSecurity & Privacy by Design: apply least-privilege, secrets management, encryption in transit and at rest, and data masking/anonymisation where required.\n\nDrive architectural decisions and define integration best practices\n\nQUALIFICATIONS/EXPERIENCE:\n\nExtensive hands-on experience (typically 6+ years) in data engineering, integration or streaming roles with demonstrable production experience.\n\nProven track record building and operating streaming platforms (Kafka) and API-based integrations, with strong Python/Java and enterprise databases and query languages skills.\n\nStrong analytical thinking, curiosity about data, attention to detail, structured problem solving and ownership - able to figure things out and drive topics to completion.\n\nSubmit your CV to:  and Subject line Role title","description_format":"text","description_chars":4973,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-11-05T13:02:45Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":29,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/imizizi-data-engineer-expert","json_url":"https://alion.io/job/imizizi-data-engineer-expert.json","meta":{"generated_at":"2026-11-07T15:40:28Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":21420,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4496229},"rest":"https://alion.io/mcp/rest/get_company?id=4496229"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fimizizi-data-engineer-expert"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fimizizi-data-engineer-expert"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fimizizi-data-engineer-expert"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/imizizi-data-engineer-expert\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fimizizi-data-engineer-expert"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3805616}},{"tool":"search_jobs","arguments":{"company":"imizizi.co.za","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3805616},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}