{"id":3874708,"url":"https://alion.io/job/pulserise-technologies-pythonsparkai-developer-tt","title":"Python/Spark/AI Developer_TT","company":{"id":4485907,"name":"PulseRise Technologies","domain":"pulserisetechnologies.io","url":"https://alion.io/company/pulserise-technologies","size_band":null,"is_staffing_agency":false,"employer_type":"staffing","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":{"label":"UTC+2","utc_offset_min":2,"utc_offset_max":2},"hiring_geo_confidence":"explicit","locations":["Ukraine"],"countries":["UA"],"hiring_countries":["ZA"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache HTTP Server","optional":false},{"name":"Azure","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Docker","optional":false},{"name":"FastAPI","optional":false},{"name":"GDPR","optional":false},{"name":"Kubernetes","optional":false},{"name":"llama.cpp","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"MS SQL","optional":false},{"name":"Mypy","optional":false},{"name":"Ollama","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Poetry","optional":false},{"name":"Pydantic","optional":false},{"name":"pySpark","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"structlog","optional":false},{"name":"Trino","optional":false},{"name":"vLLM","optional":false}],"status":"live","first_seen_at":"2026-11-06T15:41:52Z","employer_posted_date":null,"last_verified_at":"2026-11-06T15:41:52Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"We're looking for a mid-to-senior Python data engineer to join the data team of a fast-growing, privacy-focused fintech (Series A). The company is backed by a global credit bureau and a major retailer, and has teams across South Africa, the UK, the US, the UAE and Brazil.\n\nIts platform lets banks and lenders work with retailer and telco data without raw personal data ever changing hands. The main goal is to help people with no credit history get access to credit.\n\nYour purpose in this role is to lead the move of legacy SQL Server / Azure Synapse pipelines onto open-source Apache Spark and Delta Lake. You'll write production software, not notebook scripts.\n\nLogistics\n\nLocation: fully remote, Europe \n\nWorking hours: South Africa time zone (UTC+2, the same as Kyiv and one hour ahead of Warsaw). Senior leadership is on US Eastern time.\n\nEmployment: full-time. Contract type (B2B or permanent) and salary band to be confirmed.\n\nLevel: Mid-Senior, about 4–8 years of experience\nWhat you'll do\n\nRebuild legacy T-SQL stored-procedure logic as typed, tested PySpark pipelines\n\nProve that the new pipelines match the legacy outputs\n\nBuild FastAPI services that run and report on pipeline jobs\n\nAdd provider-neutral LLM features (local and hosted models) to the data workflows\n\nWork in a spec-first codebase: design docs and ADRs, with tests and documentation shipped alongside every change\n\nCore stack\n\nPython, PySpark, Spark SQL, Delta Lake, FastAPI, pytest, Docker, T-SQL, LLM integration\n\nRequirements\n\nPython as your primary production language: modern typed Python (3.10+, ideally 3.12), with type hints that pass pyright or mypy, and Pydantic models\n\n2+ years running production Apache Spark / PySpark pipelines, using the DataFrame API and Spark SQL, partitioning and performance tuning, and with a solid grasp of the driver/executor architecture\n\nMaintainable, tested code (pytest), with tooling such as uv or Poetry and a CLI framework such as Click\n\nBased in Europe or South Africa and happy to work UTC+2 hours\n\nStrong advantages\n\nExperience migrating SQL Server / Synapse / stored-procedure logic to Spark and proving output parity with the legacy system\n\nOpen-source or self-managed Spark (Spark on Kubernetes, EMR, Dataproc, Spark Connect), not only Databricks notebooks; Hive Metastore, Trino, Apache Ranger\n\nFastAPI services with request validation, HMAC-signed auth, idempotency and job-status semantics\n\nLLM features: prompt construction, structured-output validation, local models (Ollama, vLLM, llama.cpp), evaluation and guardrails\n\nPrivacy engineering: PII tokenisation and hashing, k-anonymity, GDPR/POPIA, multi-tenant isolation\n\nFintech, banking, credit-bureau, adtech-measurement or retail-data background\n\nObservability: structlog, OpenTelemetry, OpenLineage\n\nAzure (ADLS, Synapse) and a Databricks Certified Spark Developer credential","description_format":"text","description_chars":2864,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"South Africa","iso":"ZA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Outsourcing & Dedicated Teams","IT Staffing & Staff Augmentation"],"lifecycle":[{"event":"open","at":"2026-11-06T15:41:52Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":27,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-11-08T05:47:12Z"},"pay":null,"html_url":"https://alion.io/job/pulserise-technologies-pythonsparkai-developer-tt","json_url":"https://alion.io/job/pulserise-technologies-pythonsparkai-developer-tt.json","meta":{"generated_at":"2026-11-08T07:48:04Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16973,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-09T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4485907},"rest":"https://alion.io/mcp/rest/get_company?id=4485907"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpulserise-technologies-pythonsparkai-developer-tt"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpulserise-technologies-pythonsparkai-developer-tt"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpulserise-technologies-pythonsparkai-developer-tt"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/pulserise-technologies-pythonsparkai-developer-tt\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpulserise-technologies-pythonsparkai-developer-tt"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3874708}},{"tool":"search_jobs","arguments":{"company":"pulserisetechnologies.io","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3874708},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}