Lakehouse implementation Engineer

Job description:

• Own the end-to-end architecture and technical roadmap for the Lakehouse platform, including data products, data marketplace, knowledge layer, and agentic workloads.
• Define target architecture, reusable frameworks, scalability, security, performance, and operationalisation patterns for RAG, unstructured data, real-time and agentic workloads.
• Partner with business and technology teams on data contracts, SLAs, data quality, and solution delivery.
• Ensure delivery meets Bank software engineering, quality, and governance standards.
• Lead technology evaluation through RFPs/POCs, software integration, and solution design reviews.
• Drive performance optimisation, continuous improvement, and technical documentation.

Requirements

• 10–15 years’ experience implementing Data Lakehouse platforms, preferably in Financial Services, using Databricks, Snowflake, Cloudera, AWS, Azure, GCP, or similar.
• Strong experience with large-scale distributed data platforms and performance optimisation, including Iceberg/Hudi/Delta Lake, object/tiered storage, Trino/Denodo/Dremio, and distributed query engines.
• Experience designing MPP/distributed workloads across on-premise, hybrid, and cloud environments.
• Expertise in RAG and agentic workloads, including embeddings, Vector DB, Graph DB, prompt engineering, and context management.
• Strong knowledge of hybrid/cloud architecture, private connectivity, workload placement, egress optimisation, and Infrastructure-as-Code.
• Experience building and serving data products through APIs, pub/sub, real-time dashboards, generative BI, and data marketplaces.
• Experience with DevOps/engineering tools such as Jenkins, JIRA, Git, CI/CD pipelines, SonarQube, Terraform, monitoring, Control-M/Airflow, and testing tools.

Skills

Data Products, Cloudera, Continuous Improvement, Azure DevOps, Data Governance, Data Quality Assurance, Data Mining, Python, Data Science, Data Contracts