Lead Data & AI Engineer

Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!

Grow your career internally at Qualys, our best talent comes from within!

Job Description

Design, build, and scale enterprise-grade Data & AI/GenAI solutions that leverage enterprise data platforms and business applications to automate processes, enhance decision-making and deliver measurable business outcomes. This role focuses on defining and implementing the foundational Data & AI strategy for the CIO organization by combining modern data platforms, AI technologies, enterprise systems, APIs, and analytics in a secure, scalable, and production-ready environment. The ideal candidate brings experience building Data Lake or Lakehouse architectures from scratch and implement AI solutions on top of it.

Responsibilities

• Architect and build enterprise data lake/lakehouse from scratch - including ingestion, storage, transformation, serving, and governance layers - on a modern cloud platform (Snowflake, Databricks, or equivalent).
• Design and implement a medallion (Bronze/Silver/Gold) architecture to serve structured, semi-structured, and unstructured data across multiple business domains.
• Build scalable, reliable batch and streaming data pipelines to ingest data from enterprise applications and product source systems.
• Define and enforce data quality frameworks, data contracts, and SLAs across all ingestion and transformation layers.
• Implement data governance, lineage tracking, and metadata management - ensuring every data asset is discoverable, trusted, and auditable.
• Design and manage access control across the lake - including role-based access control (RBAC), row-level and column-level security - harmonizing identity and entitlements across enterprise source systems.
• Own the Data Lake platform evaluation and produce a recommendation covering cost, governance, AI readiness, and operational overhead.
• Build AI-powered applications on top of the data platform - including natural language interfaces to enterprise data, RAG-based intelligent search, automated reporting, and workflow agents.
• Integrate LLM APIs (OpenAI, Anthropic Claude, AWS Bedrock, or equivalent) into production data workflows and business-facing applications.
• Assess and recommend AI use cases across business functions and build or enable the data infrastructure required to support them.
• Evaluate build-vs-buy decisions for AI capabilities and advise leadership on platform and tooling choices.
• Collaborate with business and enterprise application teams to identify and deliver high-impact data and AI use cases.
• Develop reusable data frameworks, engineering standards, and best practices; mentor junior data engineers.

Required Qualifications

• 7-12+ years of hands-on data engineering experience, with at least 2-3 years building enterprise AI solution.
• Strong experience on enterprise Data Architectures, building Data Lakes/Lakehouse, semantic layers, data pipelines, governance, and AI-ready data foundations.
• Deep expertise in at least one modern cloud data platform: Snowflake, Databricks, or equivalent (Delta Lake, Apache Iceberg).
• Strong expertise in Python, backend development, APIs, system integration, and workflow orchestration.
• Hands-on experience with GenAI or LLM-based applications in a production context - RAG pipelines, LLM API integration, agentic workflows, NL-to-SQL, or LLM evaluation frameworks.
• Hands-on cloud infrastructure experience on AWS, Azure, GCP
• Experience building batch and/or streaming pipelines at meaningful scale (multi-TB daily volumes or higher).
• Solid understanding of data modelling - dimensional modelling, medallion architecture, data vault - and the ability to choose the right pattern for the use case.
• Experience with data governance, access control (RBAC, row-level security, column masking), and lineage tracking in a multi-source enterprise environment.
• Strong SQL - query optimization, performance tuning, and large-scale analytical workloads.
• Ability to communicate technical strategy clearly to non-technical stakeholders and leadership; experience making the business case for data platform investments.

Preferred Qualifications

• Experience integrating enterprise applications as data sources into a Data Lake/Lakehouse.
• Experience integrating AI into enterprise applications and business workflows.
• Experience building reusable AI frameworks, accelerators, or enterprise AI platforms.
• Experience with vector databases, embeddings, semantic search, and advanced RAG architectures.
• Experience with real-time or near-real-time streaming technologies
• Familiarity with dbt for transformation layer engineering.
• Experience with data cataloguing and lineage tools.
• Prior experience in a product or SaaS company, cybersecurity, or high-growth technology environment.

Educational Qualification

BE/B.Tech/MCA, preferably in Computer Science, Information Technology, or a related field.