Data Engineer - REF7025

Creative Web Technologies Pvt Ltd is a Sri Lankan software organization focused on developing innovative Data and AI solutions, empowering businesses and clients across local and international markets.

About the role:

As a data engineer, you will play a vital role in designing, developing, and managing the data architecture and tools required for analysing large data volumes. Collaborating closely with the team and reporting to the project lead, you will be taking initiative in various data engineering aspects such as designing efficient data pipelines to optimizing database performance. Your responsibilities include supporting the team in data architecture design, ETL processes, and database management.

What you will be doing:

• Design and implement scalable, high-performance data architectures to meet the organization's data processing and storage needs.
• Collaborate with teams to understand data requirements and design solutions that support analytics, reporting, and machine learning.
• Develop and maintain scalable, and efficient data pipelines for ingesting, processing, and transforming large volumes of data.
• Implement ETL (Extract, Transform, Load) processes to move and transform data between systems.
• Create, modify, and troubleshoot SQL stored procedures to meet functional and performance requirements, ensuring data accuracy and consistency.
• Manage and optimize databases, ensuring data integrity, security, and performance.
• Select and implement appropriate database technologies based on project requirements.
• Work with big data technologies to process and analyze large datasets.
• Implement distributed computing solutions for handling big data challenges.
• Establish and enforce data quality standards and best practices.
• Implement data governance policies to ensure compliance and security.
• Deploy and manage infrastructure for data storage, processing, and retrieval.
• Monitor and optimize the performance of data systems.
• Co-operate with teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver solutions.
• Communicate technical concepts and solutions to non-technical stakeholders.

What we expect:

• Bachelor's or master's degree in computer science, Information Technology, or a related field.
• 2-3 years of proven experience as a Data Engineer or in a similar role.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with database technologies (e.g., SQL, NoSQL), data warehousing, and big data technologies.
• Familiarity with cloud platforms (e.g., Azure).
• Strong problem-solving and analytical skills.

•Excellent communication and collaboration skills.