{"id":3942191,"url":"https://alion.io/job/hire-resolve-pythonsparkai-developer","title":"Python/Spark/AI Developer","company":{"id":4513499,"name":"Hire Resolve","domain":"hireresolve.za.com","url":"https://alion.io/company/hire-resolve-5","size_band":null,"is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"junior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Cape Town, South Africa"],"countries":["ZA"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":750000,"max":1050000,"currency":"ZAR","period":"year","gross":true,"usd_annual":63614},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Hudi","optional":false},{"name":"Apache Iceberg","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Docker Compose","optional":false},{"name":"FastAPI","optional":false},{"name":"Git","optional":false},{"name":"GitHub","optional":false},{"name":"MS SQL","optional":false},{"name":"Mypy","optional":false},{"name":"Poetry","optional":false},{"name":"Pydantic","optional":false},{"name":"pySpark","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Azure","optional":true},{"name":"Docker","optional":true},{"name":"GDPR","optional":true},{"name":"LLM","optional":true},{"name":"OpenTelemetry","optional":true},{"name":"Structured Outputs","optional":true},{"name":"Tokenization","optional":true},{"name":"Trino","optional":true}],"status":"live","first_seen_at":"2026-11-03T07:01:07Z","employer_posted_date":null,"last_verified_at":"2026-11-03T07:01:07Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"Description\nA company that runs a privacy-preserving data collaboration platform is seeking a AI Developer who will replatform legacy T-SQL systems onto modern Spark/Delta Lake pipelines, build robust FastAPI services around them, and integrate AI features.\n\nResponsibilities:\nData Engineering: Replatform T-SQL into Spark SQL and build production PySpark pipelines on Delta Lake.\n\nAPI Development: Build secure, idempotent FastAPI services with robust job-status semantics and authentication.\n\nQuality and Validation: Write type-hinted, tested Python (pytest) with CI gates; validate migrated pipelines against legacy systems with parity evidence.\n\nSpec-First Delivery: Work autonomously from design docs and ADRs, keeping documentation current alongside code changes.\n\nMinimum Requirements:\nExperience: 4+ years in professional Python development, with 2+ years building production Spark/distributed data pipelines.\n\nEducation: Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.\n\nCertifications: Databricks Certified Developer for Apache Spark (or equivalent) is advantageous.\n\nPython (3.12+): Modern idiomatic Python, strict static analysis (mypy/pyright), Pydantic, ABC patterns, modern packaging (uv/Poetry), and CLI tooling.\n\nApache Spark / PySpark: Production OSS Spark experience (DataFrame API, Spark SQL, tuning, driver/executor architecture, Spark Connect).\n\nLakehouse Formats: Delta Lake (or Iceberg/Hudi) including MERGE INTO, schema evolution, time travel, and idempotent writes.\n\nSQL: Advanced, dialect-portable SQL skills to translate complex legacy T-SQL logic into Spark SQL.\n\nTesting and CI/CD: Pytest (fixtures, tiered suites), Git/GitHub flow, PR-driven workflows with linting (ruff) and type-check gates.\n\nAPIs and Containers: FastAPI development and daily development using Docker Compose stacks.\n\nPreferred SkillsAI/LLM Engineering: Provider-neutral LLM integration, structured output validation, local/hosted inference, and guard-railing.\n\nGovernance and Security: PII tokenization, hashing schemes, and data privacy awareness (POPIA/GDPR).\n\nCloud and Platform: Azure data estate (ADLS Gen2, Key Vault), Trino, Hive Metastore, and OpenTelemetry observability.\n\nBenefits:\nCompetitive salary based on experience (salary can potentially be more based on experience/skills)\nIF you meet the above requirements and want to make a career-changing move, apply today by emailing your CV to \n\nPackage & Remuneration\nR750 000 - R1 050 000 per annum\nAbout Us\nHire Resolve is a top recruitment agency in South Africa, connecting businesses with talented professionals across a wide range of industries and locations.\n\nWe help companies recruit and hire skilled professionals across Engineering, Mining, IT, Finance, Manufacturing, Construction, Supply Chain, Logistics, Renewable Energy, Sales, Marketing and more. Our recruitment expertise spans entry-level, specialist, senior management and executive positions.\n\nWith recruitment solutions covering major South African cities including Johannesburg, Cape Town, Pretoria, Durban, Gqeberha and Bloemfontein, as well as locations nationwide, we help businesses find the right talent wherever they need it.\n\nWhether you're hiring for a single position or building an entire team, Hire Resolve provides tailored recruitment solutions designed around your hiring needs.\n\nReady to find your next hire? Get in touch with Hire Resolve today.","description_format":"text","description_chars":3462,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-11-08T15:57:45Z"}],"visa":[],"liveness":{"score":51,"band":"ok","label":"Likely open","p_open":1,"p_active":0.513,"p_room":1,"age_days":5,"expected_fill_days":22,"reasons":["seen:5","agency","win:early","comp:junior"],"computed_at":"2026-11-08T17:46:26Z"},"pay":{"stated_usd_annual":63614,"is_top_pay":false},"html_url":"https://alion.io/job/hire-resolve-pythonsparkai-developer","json_url":"https://alion.io/job/hire-resolve-pythonsparkai-developer.json","meta":{"generated_at":"2026-11-08T17:46:26Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":37296,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-09T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4513499},"rest":"https://alion.io/mcp/rest/get_company?id=4513499"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhire-resolve-pythonsparkai-developer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhire-resolve-pythonsparkai-developer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhire-resolve-pythonsparkai-developer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/hire-resolve-pythonsparkai-developer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhire-resolve-pythonsparkai-developer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3942191}},{"tool":"search_jobs","arguments":{"company":"hireresolve.za.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3942191},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}