AI Data Engineer

The AI Data Engineer will play a pivotal role in designing and implementing advanced data pipelines and generative AI solutions. You will be responsible for bridging the gap between raw data and actionable insights by leveraging cloud-native tools and modern AI frameworks. This role focuses on building scalable architectures that support sophisticated analytics and automated decision-making processes.

Our client is a prominent global leader in the computer software and analytics industry, known for driving digital transformation for large-scale enterprises. They foster a culture of innovation and continuous learning, providing employees with the opportunity to work on cutting-edge technologies. The organization is committed to delivering high-impact solutions that solve complex business challenges for clients worldwide.

Responsibilities

• Design and maintain robust ETL/ELT pipelines to process large-scale datasets using Databricks and PySpark.
• Develop and deploy generative AI solutions, including integration with Azure OpenAI and Microsoft Copilot.
• Execute complex migrations from legacy reporting platforms to Power BI.
• Implement data governance and quality frameworks using Collibra and Microsoft Purview.
• Collaborate with cross-functional teams to translate business requirements into technical data solutions.
• Optimize data models and SQL queries to ensure high performance and scalability.
• Provide technical expertise in the Banking and Financial Services domain to drive domain-specific AI initiatives.
• Ensure data security and compliance standards are met across all cloud-based data architectures.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related quantitative field.
• Proven experience in data engineering with strong proficiency in Python, SQL, and PySpark.
• Hands-on experience with Databricks and Azure cloud services.
• Deep understanding of Generative AI, Azure OpenAI, and Microsoft Copilot implementations.
• Expertise in Power BI development and experience in Tableau-to-Power BI migration projects.
• Familiarity with data governance tools such as Collibra and Microsoft Purview.
• Strong domain knowledge in Banking or Financial Services.
• Ability to work effectively in a full-time office environment.