Senior Data Engineer

We are a leading Software as a Service (SaaS) company that specialises in the transformation of data in the US healthcare industry through cutting-edge Artificial Intelligence (AI) solutions. We are looking for Software Developers who should continually strive to advance engineering excellence and technology innovation. The mission is to power the next generation of digital products and services through innovation, collaboration, and transparency. You will be a technology leader and doer who enjoys working in a dynamic, fast-paced environment.

Responsibilities:

• Design, develop, and maintain robust and scalable ETL/ELT pipelines to ingest and transform large datasets from various sources.
• Optimise and manage databases (SQL/NoSQL) to ensure efficient data storage, retrieval, and manipulation for both structured and unstructured data.
• Collaborate with data scientists, analysts, and engineers to integrate data from disparate sources and ensure smooth data flow between systems.
• Implement and maintain data validation and monitoring processes to ensure data accuracy, consistency, and availability.
• Automate repetitive data engineering tasks and optimise data workflows for performance and scalability.
• Work closely with cross-functional teams to understand their data needs and provide solutions that help scale operations.
• Ensure proper documentation of data engineering processes, workflows, and infrastructure for easy maintenance and scalability.

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 3 - 9 years of hands-on experience as a Data Engineer or in a related data-driven role.
• Strong experience with ETL tools like Apache Airflow, Talend, or Informatica.
• Expertise in SQL and NoSQL databases (e. g., MySQL, PostgreSQL, MongoDB, Cassandra).
• Strong proficiency in Python, Scala, or Java for data manipulation and pipeline development.
• Experience with cloud-based platforms (AWS, Google Cloud, and Azure) and their data services (e. g., S3 Redshift, BigQuery).
• Familiarity with big data processing frameworks such as Hadoop, Spark, or Flink.
• Experience in data warehousing concepts and building data models (e. g., Snowflake, Redshift).
• Understanding of data governance, data security best practices, and data privacy regulations (e. g., GDPR, HIPAA).
• Familiarity with version control systems like Git.