Group Data Engineer I

KEY ACCOUNTABILITIES

Solution Design & Architecture:

• Lead the design of end-to-end data platform solutions, ensuring they meet both business and technical requirements.

• Architect scalable, high-performance data platforms leveraging technologies such as cloud-based solutions (AWS, Azure, GCP), data lakes, data warehouses, ETL/ELT pipelines, and real-time data streaming.

• Design integrated solutions that can handle diverse data sources (structured, semi-structured, unstructured) and support advanced analytics, machine learning, and AI applications.

  Strategic Data Platform Planning & Roadmap:

• Define the long-term vision and roadmap for the data platform, ensuring alignment with the organization’s data strategy and business goals.

• Develop migration and modernization strategies for legacy data systems to modern data platforms, including cloud adoption and hybrid architectures.

• Evaluate emerging technologies and industry trends to recommend innovative solutions that enhance the data platform’s capabilities.

  Cross-Functional Collaboration:

• Collaborate closely with data engineers, data scientists, business analysts, and other IT teams to ensure that data architectures align with business objectives and deliver the necessary insights for decision-making.

• Work with business stakeholders to understand data requirements, translating them into technical specifications and ensuring solutions meet business needs.

• Foster collaboration across technical teams to ensure the seamless integration of systems, data sources, and workflows.

  Data Governance, Security & Compliance:

• Implement data governance frameworks, ensuring that data is accurate, consistent, and accessible across the organization while maintaining the highest levels of security and compliance.

• Develop and enforce data security policies, ensuring that data is protected in compliance with regulatory standards such as GDPR, CCPA, HIPAA, etc.

• Establish data lineage and metadata management practices to support data integrity and transparency.

  Cloud Data Architecture & Migration:

• Lead the design and implementation of cloud-based data solutions, optimizing data storage, compute, and analytics services to ensure performance, scalability, and cost-efficiency.

• Drive cloud migration projects, working with engineering teams to move on-premises data solutions to cloud platforms (Azure, AWS, GCP).

• Architect solutions for data warehousing, data lakes, and analytics, ensuring that the architecture is resilient, flexible, and cost-optimized.

  Performance Optimization & Scalability:

• Ensure that the data platform is capable of handling large volumes of data and providing low-latency access for real-time analytics and reporting.

• Continuously assess and optimize the performance, scalability, and cost-efficiency of the data architecture.

• Lead the design of systems that scale efficiently, both vertically and horizontally, to accommodate growing data needs.

  Leadership & Mentoring:

• Provide technical leadership to the data engineering and architecture teams, ensuring best practices and high standards are maintained in solution design and implementation.

• Mentor junior team members, offering guidance on architectural design, data modelling, and the latest data technologies.

• Promote a culture of continuous learning, encouraging team members to stay up to date with industry trends and innovations.

  Solution Implementation & Delivery:

• Oversee the implementation and delivery of data platform solutions, ensuring they are deployed successfully and meet technical specifications.

• Troubleshoot and resolve any issues related to the data architecture and platform.

• Ensure solutions are delivered on time and within budget, meeting both functional and non-functional requirements.

  Documentation & Reporting:

• Create and maintain comprehensive documentation for data architecture, solution designs, and technical processes.

• Produce regular status reports and updates to senior management, highlighting key milestones, risks, and opportunities.

QUALIFICATIONS, EXPERIENCE AND SKILLS

Qualifications:

• Bachelor’s or master’s degree in computer science, Engineering, Information Technology, or a related field.
• Minimum of 7+ years of experience in data architecture, data engineering, or a similar role, with a strong focus on designing large-scale data platforms.
• Proven experience in architecting cloud-based data solutions, including data lakes, data warehouses, ETL pipelines, and analytics platforms (Azure, AWS, GCP).
• Strong knowledge of data modelling, data governance, data security, and cloud-native data technologies.
• Experience with data integration techniques, including ETL/ELT processes, real-time data streaming, and batch processing.
• Expertise in big data tools (e.g., Hadoop, Spark), database systems (e.g., SQL, NoSQL), and data warehousing platforms.
• Strong understanding of data privacy, security, and compliance frameworks (e.g., GDPR, HIPAA).
• Experience in leading data migration projects and modernizing legacy data systems.

Key Skills:

• Strong leadership, collaboration, and communication skills.
• Expertise in cloud platforms and services (Azure preferred).
• Proficiency in data pipeline orchestration tools (e.g., Apache Airflow, Azure Data Factory).
• Knowledge of containerization and microservices architecture.
• Familiarity with data visualization and BI tools (e.g., Power BI, Tableau).
• Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
• Ability to think strategically while balancing business needs and technical solutions.
• Experience with Agile methodologies and working in a fast-paced, collaborative environment.

Desirable Qualifications:

• Certifications such as Microsoft Certified: Azure Solutions Architect Expert or Google Cloud Professional Data Engineer.
• Experience with machine learning and AI workloads on data platforms.
• Knowledge of DevOps practices and CI/CD for data pipelines.