Data Engineer - Snowflake Cortex

Position : Data Engineer - Snowflake Cortex - AWS Glue

Purpose of the Position :

To lead the data engineering team in architecting and delivering enterprise-scale Snowflake solutions for retail analytics, ensuring alignment with strategic business goals and modernization initiatives. The role will also drive Snowflake Cortex AI development, building AI agents and GenAI use cases that turn governed retail data into conversational, insight-driven experiences for business users.

Location : Chennai / Bangalore / Pune / Nagpur

Key Result Areas and Activities :

- Data Pipeline Development : Drive ingestion and transformation of multi-source retail data (MMS, CRM, Workday, Courion etc).

- Architecture Leadership : Define and implement Snowflake architecture for multi-layered retail data processing.

- ETL Strategy : Lead migration strategy from Talend to AWS Glue, ensuring automation and scalability.

- ETL Development : Build and optimize ETL workflows using Talend, AWS Glue and SQL stored procedures.

- Strategic Alignment : Align data engineering practices with retail business priorities (sales, customer, workforce, inventory).

- Data Modelling : Create and maintain Snowflake tables and layered views for business consumption.

- Cortex Agent Creation : Build, configure and deploy Cortex Agents that use Cortex Analyst (semantic models/views) for structured data and Cortex Search for unstructured content, with Streamlit in Snowflake as the user-facing conversational interface.

- Cortex Use-Case Delivery : Identify, prototype, and take Cortex use cases from POC to production (e.g., natural-language sales/inventory insights, product review and customer feedback analysis, document intelligence, demand forecasting).

- AI Readiness & Semantic Layer : Prepare AI-ready data foundations, including semantic models, curated views and metadata, so Cortex-based solutions give accurate, trusted responses.

- Data Governance : Implement standards for lineage, documentation, and compliance.

- Collaboration : Coordinate with team members, work closely with data analysts, business stakeholders, and regulatory teams to understand data needs and deliver solutions.

- Performance Optimization : Ensure efficient query performance, storage management and cost-effective data processing in Snowflake.

- Documentation & Governance : Maintain clear documentation of data models, transformations, and lineage to support audit and compliance requirements.

- Team Leadership : Mentor and manage engineers across different levels, ensuring delivery excellence.

- Innovation : Explore snowflake features like dynamic tables, automation, and advanced Snowflake features for efficiency.

Essential Skills :

- Advanced Snowflake expertise (architecture, optimization, governance).

- Strong hands-on experience with Snowflake (data warehousing, performance tuning, security).

- Hands-on experience with Snowflake Cortex AI, including Cortex AI/LLM functions, Cortex Analyst, Cortex Search and Cortex Agents.

- Proven experience building and deploying Streamlit in Snowflake apps, especially conversational and agent-based interfaces.

- Experience creating semantic models/semantic views for natural-language-to-SQL use cases.

- Strong SQL skills with stored procedure development.

- Solid understanding of SQL and advanced query optimization techniques.

- Strong leadership in SQL-based ETL and stored procedure frameworks.

- Deep understanding of retail data domains and compliance frameworks.

- Proven experience in cloud-native ETL (AWS Glue, Talend).

- Experience in managing cross-functional teams and delivery pipelines.

Desirable Skills :

- Exposure to ML pipelines for retail analytics.

- Strategic experience in cost optimization and performance tuning.

- Knowledge of prompt engineering, RAG patterns, LLM evaluation and guardrails.

- Exposure to Cortex Fine-Tuning, Document AI, Snowflake ML (Model Registry, Feature Store) and Snowflake Notebooks.

- Snowflake certifications (SnowPro Core / Advanced, or GenAI-related credentials).

Qualification :

- Experience : 6 - 10 years in data engineering, with 3+ years in Snowflake leadership/managerial roles.

- Education : Bachelor's or master's degree in computer science, Data Science, Bioinformatics, or related field.

Qualities :

- Strong analytical and problem-solving mindset.

- Detail-oriented with a focus on data accuracy and compliance.

- Excellent communication and stakeholder management skills.

- Ability to work independently and collaboratively in cross-functional teams.

- Adaptability to fast-paced and regulated environments.

- Curiosity for emerging AI technologies and ability to translate AI capabilities into clear business outcomes.

Skills

Data Engineering, AWS Glue, Generative AI, SQL, Streamlit, Talend, Data Modeling, Snowflake DB, ETL