Data Engineer
[What the role is]
The National Library Board (NLB) nurtures Readers for Life, Learning Communities and a Knowledgeable Nation by promoting reading, learning and history through our network of 28 libraries across Singapore, the National Library and the National Archives of Singapore.At NLB, you can look forward to a flexible, hybrid work environment with rewarding, impactful deliverables that improve the lives of the community. With caring colleagues, you will be in an ideal environment for professional development, innovation and impact.
We can offer you:
- A chance to Impact the Community, bridge gaps and empower everyone with the opportunity for learning and discovery.
- An opportunity to Innovate and Push Boundaries. We value ideas, curiosity and a passion for innovation to transform our libraries and archives.
• An organisation that Cares for your Growth and Wellbeing through investment in staff development and wellbeing in a supportive and collaborative environment.
[What you will be working on]
Data Architecture Design: Collaborating with NLB solution architects, Data Management Platform awarded vendor and other stakeholders to understand data requirements and designing scalable and efficient data architectures.
Data Management: Working closely with the vendor to set up and manage Databases and Data Lake, including selecting appropriate database technologies, optimizing query performance, and ensuring data integrity and security.
Data Integration: Working closely with the vendor to integrate data from different sources to create a unified and consistent view of the data for analysis and reporting purposes.
Data Transformation and Manipulation: Working closely with the vendor to perform data transformations, such as masking, aggregations, joins, and filtering, to meet the specific requirements of data analysis and reporting.
Data Migration: Collaborating with the vendor on end-to-end data migration to a new data management platform, from strategizing migration plan to testing and production deployment. Reviewing migration and test strategies to ensure alignment with architectural standards and project requirements.
Data Quality and Governance: Implementing data quality checks and validation processes to ensure the accuracy, completeness, and consistency of the data. Establishing data governance practices and standards to maintain data integrity and compliance with regulations.
Monitoring and Troubleshooting: Monitoring data pipelines and systems to identify and resolve issues related to data ingestion, processing, or storage. Performing troubleshooting and debugging tasks to maintain data pipeline reliability.
Collaboration and Documentation: Collaborating with vendor and cross-functional teams to understand their data requirements and provide them with the necessary information on the tools and infrastructure. Documenting data engineering processes, technical specifications, configurations, and best practices.
Support Data Platform Operation: Undertaking maintenance support, development and enhancement work, including building new features, improving existing pipelines, and ensuring the data management platform continues to meet evolving business needs
[What we are looking for]
Relevant IT qualifications or equivalent experience with advanced qualifications.
At least 2-5 years of relevant technical working experience with data management, RDBMS and ETL, covering requirements gathering, data modelling, pipeline development, testing, deployment and post-production support.
Experience in data migration projects, including migration strategy design, data profiling and quality assessment, cutover planning, and post-migration validation.
Experience in AWS Platform Architecture which may include S3, Data Lakehouse frameworks, Apache Iceberg,, Glue, DataBrew, Lambda, Athena, etc.
Proficiency in SQL and Python.
Knowledge of the various Data Management Tools such as Databrick, Data pipes and/or Snowflake is an advantage.
Experience with Apache Airflow for orchestrating and scheduling complex data workflows and pipelines.
Knowledge in various data analytics tools like Qlik Sense, Tableau, etc is an advantage.
Able to communicate with all levels to manage relevant stakeholders and a good team player.
The work location will be at ESR BizPark @ Chai Chee. This is a project contract expiring 30 Dec 2027.