Senior DataOps Engineer, Data Platform Section - Data Platform Department (DPD)

Job Description:

Business Overview

The Technology Platforms Division (TPD) drives the growth of Rakuten's ecosystem by delivering innovative, high-quality technology platforms characterized by integrated control and strategic partnerships.

Within TPD, the Cloud Platform Supervisory Department (CPSD) develops and manages Rakuten's state-of-the-art cloud platform, empowering global scalability and accelerating innovation across its diverse business units.

Department Overview

The Data Platform Department (DPD) at Rakuten Group develops and maintains a comprehensive data platform, empowering over 70 Rakuten services with solutions for data ingestion, discovery, governance, analytics, and querying. We support data-driven decision-making across one of Japan's largest data ecosystems, providing the tools and infrastructure to support key domains such as Data Lakes, Data Warehouses, and Business Intelligence.

Position:

Position Details

As a Senior DataOps Engineer, you will lead the design and delivery of multi-cloud data pipeline infrastructure within the Data Platform Department at Rakuten Group. You will work closely with BU data engineering teams, platform leads, and governance stakeholders to define engineering standards, build reusable CI/CD templates, and drive the self-service data platform adoption across 70+ business units.

Responsibilities

· Lead the design and implementation of standardised pipeline templates for multi-cloud data ingestion, transformation, and delivery (GCP BigQuery / Dataflow and Azure Databricks)

· Build and maintain CI/CD standards and tooling (GitHub Actions / Cloud Build) used across BU data engineering teams

· Implement automated testing, validation, and rollback frameworks to enforce SLA and quality gates at the pipeline level

· Lead BU onboarding to the data platform (target: 8-week SLA), providing technical guidance and runbook documentation

· Drive adoption of Open Table Format (OTF / Apache Iceberg) as the group-wide storage standard across BU workloads

· Operate and improve Apache Airflow orchestration environments; define DAG standards, dependency graphs, and alerting

· Partner with the Data Governance team to enforce metadata and lineage capture at pipeline runtime

· Mentor junior DataOps Engineers and contribute to the team engineering knowledge base - runbooks, ADRs, and onboarding guides

Mandatory Qualifications:

· Bachelor's Degree (BS) in Computer Science, Data Engineering, or a related field, or equivalent education and experience (8 years+)

· Proficient in DataOps or data platform engineering, including CI/CD for data pipelines (5 years+)

· Proficient with GCP data services including BigQuery, Dataflow, Cloud Composer, and Pub/Sub (4 years+)

· Proficient with Azure data services including Databricks, Azure Data Factory, and Azure Data Lake Storage (3 years+)

· Proficient with Apache Spark for large-scale distributed data processing (4 years+)

· Proficient in Python for pipeline scripting, data transformation, and automation (4 years+)

· Proficient with Apache Airflow - DAG design, scheduling, backfill strategy, and SLA monitoring (3 years+)

· Proficient with CI/CD automation tools such as GitHub Actions, Cloud Build, or Jenkins (4 years+)

· Proficient with infrastructure-as-code tools such as Terraform (3 years+)

· Proficient in SQL (BigQuery SQL / SparkSQL / ANSI SQL) (5 years+)

Desired Qualifications:

· Experience with Apache Iceberg or Delta Lake open table formats and lakehouse architecture (2 years+)

· Experience in a platform engineering or developer-experience (DX) role (2 years+)

· Experience with streaming pipelines - Kafka, Dataflow Streaming, or Apache Flink (2 years+)

· Experience working in a multi-BU or enterprise-scale data platform environment (3 years+)

· Familiarity with data mesh or federated data platform operating models

#engineer #infrastructureengineer #technologyplatformdiv

Languages: