{"id":4067566,"url":"https://alion.io/job/bp-senior-data-engineer-5","title":"Senior Data Engineer","company":{"id":2557,"name":"BP","domain":"bp.com","url":"https://alion.io/company/bp-com","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":99,"open_postings":37,"ghost_share":0,"stale_share":0.243,"repost_share":0,"time_to_fill_p50_days":17,"computed_at":"2026-11-11T05:49:22Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Kuala Lumpur, Malaysia"],"countries":["MY"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":true,"has_equity":false,"technologies":[{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Git","optional":false},{"name":"IAM","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-11-11T12:08:50Z","employer_posted_date":"2026-11-11","last_verified_at":"2026-11-11T12:08:50Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Entity:\nTechnologyJob Family Group:\nIT&S GroupJob Description:\nLet me tell you about the role\nAs the primary Data Engineer within the squad, this role provides end-to-end technical ownership of the data domain. The Senior Data Engineer will design, build, migrate, operate, and continuously improve data products and pipelines, ensuring they are scalable, reliable, secure, observable, and cost-effective.\nKey Responsibilities\nLead the design, development, migration, and operation of scalable data pipelines and data products using Databricks, PySpark, Spark SQL, and AWS.\nOwn the squad’s data engineering domain, providing technical direction and ensuring solutions are reliable, supportable, secure, and aligned with business needs.\nDesign and implement scalable data ingestion and migration frameworks using technologies such as Databricks Auto Loader and Delta Live Tables (DLT).\nDevelop and maintain data models, transformation pipelines, and storage solutions with strong focus on data quality, integrity, performance, and scalability.\nBuild and maintain pipeline orchestration and deployment capabilities using Databricks Workflows, Databricks Asset Bundles (DAB), CI/CD, and Infrastructure as Code.\nIntegrate data platforms with the AWS ecosystem, including IAM, storage, security, networking, and supporting cloud services.\nImplement effective governance, security, lineage, and access controls using Unity Catalog and enterprise standards.\nEstablish observability, monitoring, alerting, and cost optimisation across data pipelines and Databricks workloads.\nDrive operational excellence through incident resolution, root cause analysis, automation, runbooks, and continuous reliability improvements.\nProvide technical leadership through architecture decisions, code reviews, engineering standards, mentoring, and knowledge sharing.\nWhat you will need to be successful (experience and qualifications)\n8+ years of experience in Data Engineering, Software Engineering, or related technical disciplines.\n5+ years of hands-on Python experience and 3+ years working with PySpark and distributed data processing.\nStrong hands-on expertise in Databricks, PySpark, Spark SQL, SQL, Delta Lake, and data modelling.\nExperience designing and operating enterprise-scale data ingestion, transformation, migration, and processing frameworks.\nHands-on experience with Databricks Auto Loader, Delta Live Tables (DLT), Workflows, and Databricks Asset Bundles (DAB).\nStrong AWS experience, including IAM, S3 and integration with AWS cloud services and infrastructure.\nExperience with Unity Catalog, including data governance, access control, metadata, and lineage.\nExperience with CI/CD, Infrastructure as Code, automated testing, Git, and deployment automation.\nStrong understanding of observability, data quality, performance optimisation, cost monitoring, reliability, and production support.\nGood understanding of information security, data governance, privacy, resilience, and operational best practices.\nStrong technical leadership, problem-solving, communication, stakeholder management, and mentoring skills.\nAbout bp\nAt bp, we provide the following environment and benefits to you:\nA company culture where we respect our diverse and unified teams, where we are proud of our achievements and where fun and the attitude of giving back to our environment are highly valued.\nPossibility to join our social communities and networks\nLearning opportunities and other development opportunities to craft your career path\nLife and health insurance, medical care package\nAnd many other benefits. We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform crucial job functions, and receive other benefits and privileges of employment.\nTravel Requirement\nNo travel is expected with this roleRelocation Assistance:\nThis role is not eligible for relocationRemote Type:\nThis position is a hybrid of office/remote workingSkills:\nCloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more}.\n\n Legal Disclaimer:\nWe are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individualswith an accessibility need may request an adjustment/accommodationrelated to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodationrelated to the recruitment process, please contact us.\nIf you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.","description_format":"text","description_chars":5697,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Health insurance","Relocation assistance"],"hiring_locations":[{"name":"Malaysia","iso":"MY","kind":"country"}],"hiring_excludes":[],"relocation_offered":true,"industries":["Fossil Fuels","Oil & Gas"],"lifecycle":[{"event":"open","at":"2026-11-11T12:08:50Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":16,"reasons":["conf:4","win:early","comp:brand"],"computed_at":"2026-11-11T17:01:11Z"},"pay":null,"html_url":"https://alion.io/job/bp-senior-data-engineer-5","json_url":"https://alion.io/job/bp-senior-data-engineer-5.json","meta":{"generated_at":"2026-11-11T17:01:11Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":32346,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-12T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2557},"rest":"https://alion.io/mcp/rest/get_company?id=2557"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbp-senior-data-engineer-5"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbp-senior-data-engineer-5"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbp-senior-data-engineer-5"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/bp-senior-data-engineer-5\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbp-senior-data-engineer-5"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4067566}},{"tool":"search_jobs","arguments":{"company":"bp.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4067566},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}