{"id":2548414,"url":"https://alion.io/job/pathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring","title":"Senior Data Engineer - Streaming and Data Platform (Contract) URGENT HIRING","company":{"id":3802179,"name":"Pathsource Consulting","domain":"path-source.com","url":"https://alion.io/company/pathsource-consulting","size_band":null,"is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"contractor","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Singapore"],"countries":["SG"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":9000,"max":14000,"currency":"SGD","period":"month","gross":true,"usd_annual":130668},"salary_estimate":null,"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Apache Hudi","optional":false},{"name":"Apache Iceberg","optional":false},{"name":"Apache Kafka","optional":false},{"name":"CI/CD","optional":false},{"name":"Dagster","optional":false},{"name":"dbt","optional":false},{"name":"Delta Lake","optional":false},{"name":"Docker","optional":false},{"name":"Git","optional":false},{"name":"Linux","optional":false},{"name":"Prefect","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Amazon S3","optional":true},{"name":"Incident Management","optional":true},{"name":"Kubernetes","optional":true},{"name":"MinIO","optional":true},{"name":"OpenTelemetry","optional":true},{"name":"Polaris","optional":true},{"name":"Prometheus","optional":true},{"name":"Time Series","optional":true},{"name":"Trino","optional":true},{"name":"VictoriaMetrics","optional":true}],"status":"live","first_seen_at":"2026-10-14T00:00:00Z","employer_posted_date":null,"last_verified_at":"2026-10-14T00:00:00Z","board_verified":false,"closed_at":null,"days_open":30,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":30},"description":"The client is a Singapore-based data consultancy building anon-premises data platform for a major data-centre operator across APAC. Each site produces telemetry from thousands of nodes at rates of up to millions of data points per second.\nThis Senior Data Engineer will build the streaming pipelines that bring telemetry into the platform and establish the delivery tooling used across the wider data-engineering team.\nThis is an initial six-month contract. Any extension will depend on project requirements and is not guaranteed.\nImmediate AvailabilityImmediate hiring priority: Priority will be given to Singapore Citizens and Permanent Residents who can start immediately or within two weeks. The role is best suited to candidates between roles, nearing the end of a contract or project, or otherwise available at short notice. Extended notice periods are unlikely to meet the project timeline.\nKey Responsibilities\n· Build and operate Kafka-to-Apache Iceberg streaming pipelines, beginning with alert history and documents and scaling across telemetry topics at three times current volume.\n· Design reliable Kafka consumers or Kafka Connect integrations with correct offset management, replay and failure recovery.\n· Run and tune Spark workloads in production across streaming and batch use cases.\n· Design and optimise Iceberg tables covering partitioning, file sizing, compaction and schema evolution.\n· Set up and maintain the team's data tooling, including dbt, Dagster or a comparable orchestrator, CI/CD, data-quality checks, lineage and documentation.\n· Build feature tables and training datasets for the client's machine-learning team.\n· Monitor data freshness, volume, performance and failures, and participate in the production on-call rotation.\n· Pair with the client's data engineers and prepare clear runbooks for operational handover.\nRequirements\n· Four to seven or more years of experience building and operating production data pipelines.\n· Strong hands-on Python and SQL skills.\n· Production experience with Kafka consumers or Kafka Connect, including offset management and failure recovery.\n· Experience running and tuning Spark jobs in production for streaming or batch workloads.\n· Experience designing tables in Apache Iceberg, Delta Lake or Apache Hudi, including partitioning, file sizing, compaction and schema evolution.\n· Experience with petabyte-scale data ingestion and performance tuning.\n· Hands-on experience implementing dbt, an orchestrator such as Dagster, Airflow or Prefect, and CI/CD for a data-engineering team.\n· Production on-call and incident-management experience, including incident documentation.\n· Strong working knowledge of Linux, Docker and Git.\n· Shortlisted candidates must complete a hands-on technical test\nPreferred Experience\n· Experience running data systems on-premises or on self-managed infrastructure.\n· Experience building ingest-level data-quality checks and monitoring pipeline freshness, volume and failures.\n· Experience with high-volume time-series or observability data using Prometheus, VictoriaMetrics or OpenTelemetry.\n· Experience with Trino or a comparable distributed SQL engine.\n· Experience building feature tables or training datasets for machine-learning teams.\n· Exposure to Kubernetes, self-hosted S3-compatible storage such as MinIO or Ceph, Apache Polaris and OpenLineage.\nContract and Work Arrangement\n· Initial six-month contract with the Singapore-based data consultancy.\n· Open to Singapore Citizens and Permanent Residents only.\n· Based in Singapore, with office attendance expected three to five days per week.\n· Travel to client sites in Australia or elsewhere in the region may be required for site go-lives.\n· Participation in the production on-call rotation is required and may involve work outside normal business hours.\nHiring ProcessPlease note: The hands-on technical assessment is a mandatory selection stage.\n1. Technical assessment and interview based on a practical problem drawn from the role.\n2. Team interview covering engineering approach, code review, delivery and incident handling.\n3. Client interview with the client's engineering team.\nHow to ApplyEmail your updated resume in MS Word format to Wills Heng Wei He (EA Personnel Registration No. R1549842) .\nFor Job Seekers Free Resume ReviewGet a FREE PathSource Resume Score and review report at resume.path-source.com. With your consent, you may also join PathSource Consulting's talent network for suitable future opportunities.\nOptional career resource only. Participation is not required and does not affect consideration for this vacancy.\nFor Immediate ConsiderationPlease indicate the following information in your resume:\n· Current and expected salary\n· Reason or reasons for leaving\n· Earliest available commencement date\n· Confirmation that you are prepared to complete the mandatory technical assessment\nBy submitting an application or resume to us, you consent to PathSource Consulting collecting, using, retaining and disclosing your personal information to prospective employers for recruitment purposes.\nSkills\nBusiness Data Analysis, Data Ingestion, Airflow, Apache Spark, Data Pipeline, Apache POI, Workload Automation, Sizing, Production Support, Telemetry, Apache Kafka, SQL Development, Offset, data integration projects, Incident Management, Data Administration","description_format":"text","description_chars":5371,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Singapore","iso":"SG","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-14T16:28:30Z"}],"visa":[],"liveness":{"score":32,"band":"fade","label":"Fading","p_open":0.85,"p_active":0.503,"p_room":0.75,"age_days":29,"expected_fill_days":30,"reasons":["seen:29","agency","velocity","win:late"],"computed_at":"2026-11-12T10:38:37Z"},"pay":{"stated_usd_annual":130668,"is_top_pay":false},"html_url":"https://alion.io/job/pathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring","json_url":"https://alion.io/job/pathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring.json","meta":{"generated_at":"2026-11-13T13:08:01Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":26316,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-14T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3802179},"rest":"https://alion.io/mcp/rest/get_company?id=3802179"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/pathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpathsource-consulting-senior-data-engineer-streaming-and-data-platform-contract-urgent-hiring"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2548414}},{"tool":"search_jobs","arguments":{"company":"path-source.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2548414},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}