Data Engineer - BI Insights Group, Data Platform Department (DPD)

Job Description:

Department Overview

The Data Platform Department (DPD) at Rakuten Group develops and maintains a comprehensive data platform, empowering over 70 Rakuten services with solutions for data ingestion, discovery, governance, analytics, and querying. We support data-driven decision-making across one of Japan's largest data ecosystems, providing the tools and infrastructure to support key domains such as Data Lakes, Data Warehouses, and Business Intelligence.

Position:

Position Details

・Hands-on design, development, testing, and implementation of scalable, cloud-based data warehouse solutions, optimized data pipelines, and robust data modeling

・Maintain a healthy data warehouse ecosystem through comprehensive infrastructure management (system logs, storage, compute resources, batch scheduling, etc.), ensuring seamless data integration and proactively addressing performance bottlenecks

・Internal Tooling: Adapt to and utilize Rakuten’s internal data pipeline orchestration framework, built on top of GCP-managed Airflow

・Communication: Collaborate with cross-functional teams to identify data requirements and align them with business objectives

・Documentation: Maintain comprehensive documentation, including platform architecture, data dictionaries, infrastructure configurations, operational processes and so on

・ Collaborate with Stakeholders: Work closely with end-users to understand requirements, gather feedback, and deliver impactful, customized reports

・ Solution Design: Propose and present visualization solutions to relevant stakeholders

・ Data Modeling & Mart Design: Architect reporting-specific data marts to ensure high-performance dashboards

・ Data Pipeline Management: Develop, maintain, and optimize pipelines, including DAG scheduling, data ingestion, and ETL/ELT processes to ensure data is "collected from" and "delivered to" efficiently and reliably

・ Data Quality Assurance: Validate and ensure the accuracy, consistency, and reliability of all data used in visualizations

・ Frontend Development: Design, build, and manage interactive reports using modern data visualization tools.

Mandatory Qualifications:

・ B.S. in Computer Science, or a related field, or equivalent education and experience

・ 3+ years of end-to-end experience in data engineering (SQL) and operational experience in cloud-based or Linux systems

・ Experience in developing and building robust data warehouse ecosystems using Google Cloud Platform (GCP - BQ, GCS), Python, and shell scripting

・ Understanding of data warehouse/big data concepts, data modeling, batch-processing pipelines (ETL), and data integration techniques for managing large volumes of data

・ Hands-on experience creating and leveraging data visualization platforms (e.g., DOMO, Tableau, MicroStrategy, PowerBI, etc.)

・ Experience with Git

Desired Qualifications:

・Experience with Atlassian tools (Confluence, Jira, BitBucket)

・Experience working on streaming data pipelines using tools such as Airflow, Apache Kafka, or Spark Streaming

・Experience managing systems with high Service Level Objectives/Agreements (SLOs/SLAs)

・ Effective communication and project management skills

・ Quick learner, highly adaptable, and committed to continuous learning

・ Strong problem-solving and solution-creation mindset

・ Detail-oriented and driven by excellence

・ A "Can-Do" attitude and willingness to embrace new challenges

・ Strong teamwork and collaboration capabilities

#engineer #DataEngineer #technologyplatformdiv