Senior Data Engineer
[What the role is]
The mission of Housing & Development Board (HDB) is to provide affordable, quality housing and a great living environment where communities thrive. To achieve its mission, HDB aims to be data-driven to the core and adopt evidence-based decision making in developing better policies, improving service delivery, and optimising operations.[What you will be working on]
Data Pipeline Infrastructure & Architecture
Design and implement scalable data architectures on cloud data platforms with high availability, security, and performance
Lead development of Data Lakehouse solutions
Collaborate with stakeholders to understand requirements and translate them into technical specifications
Pipeline Development & Optimisation
Build andmaintainrobust ETL/ELT pipelines using modern data engineering tools and frameworks
Optimise data processing workflows for performance, cost-effectiveness, and reliability
Implement automated data quality checks and monitoring systems to ensure data integrity
Data Systems Architecting & Solutioning
Design and architect comprehensive cloud-native Data & AI solutions aligned with businessobjectivesand technical requirements
Lead cloud migration strategies and oversee implementation of complex multi-cloud environments
Drive innovation through integration of Data & AI capabilities into HDB’s Data & AI platform product architectures
Conduct technical assessments and recommend modernised approaches using cloud native technologies
Maintain architectural documentation
Cloud Platform Operations
Leverage Cloud Native Services to build and manage data infrastructure
Implement infrastructure as code practices using Terraform
Ensure compliance with security standards and data governance policies
Technical Leadership & Collaboration
Mentor junior data engineers andprovidetechnical guidance on complex challenges
Participate in architectural reviews and contribute to data strategy evolution
[What we are looking for]
Bachelor’s degree in computer science, Information Technology, Computer Engineering, or related field
Minimum 3 years of relevant experience in data systems architecture, data systems integration, and data pipeline setup at production scale
Good understanding of cloud computing principles including infrastructure as code, containerisation, microservices architecture, cloud security frameworks, identity and access management, network architecture, and distributed systems
Proven ability to translate business requirements into technical solutions
Excellent communication skills for presenting complex concepts to diverse audiences
Experience with cloud security frameworks, compliance requirements, and risk management
Experience in data domains (e.g.DataOps, Data Lakehouse) and AI/ML Domains (e.g.MLOps,LLMOps)
Strong Knowledge and Hands-on experience with SQL, Python and Apache Spark
Hands-on experience with Apache Kafka, Airflow, or similar technologies
Good to Have:
Proficiencyin Amazon Web Services (AWS) services
Relevant cloud certifications (e.g. AWS Solutions Architect Professional, AWS Data Engineer Associate) would be an advantage
Experience with Data & AI cloud-native services (e.g. Amazon SageMaker Unified Studio, Amazon Quick Suite, AWS S3, AWS Glue, AWS Lake Formation, AWS Bedrock, AWS Agent Core).
Familiarity with serverless computing, edge computing, and IoT architectureswould be an advantage.
Experience with machine learning operations (MLOps) and ML model deployment pipelines
Knowledge of data governance frameworks and metadata management tools
Familiarity with data visualisation tools and business intelligence platforms