Senior Software Engineer

CORE PROFILE

Reporting to the Enterprise AI Manager, the Senior AI Engineer will be responsible for architecting, building, and deploying production-grade AI systems that leverage large language models (LLMs) and agentic architectures to automate enterprise processes. The role focuses on designing reliable agent systems, building evaluation frameworks for LLM outputs, and integrating stack-agnostic AI capabilities across enterprise platforms and tools. Working knowledge of agent orchestration tools (e.g. n8n, Dify, Hermes) and cloud platforms (e.g. Azure, AWS) is a plus, with emphasis on production reliability, governance, and BSP regulatory compliance.

Unlike a traditional ML Engineer who trains models from scratch, this role emphasizes deep understanding of how LLMs behave in production, how to architect multi-step agentic workflows that are trustworthy and auditable, and how to guide the broader Enterprise AI team on prompt engineering, tool design, and AI evaluation best practices. The Senior AI Engineer will serve as the team's technical authority on AI/LLM systems and will directly shape Maya's enterprise AI strategy.

NATURE OF WORK

Agentic AI Solution Design and Development

• Design and build Agentic AI solutions using existing LLMs and integrate them with Enterprise Applications/Systems where applicable.
• Architect multi-step agent workflows with reliable tool/function calling, memory management, error handling, and graceful degradation
• Build and maintain AI solutions for document processing (IDP), process automation, and internal knowledge management using agentic patterns
• Actively participate in coding tasks, including building and deploying AI/ML models into production environments, and integrating solutions within the current tech stack

LLM Evaluation & Reliability Engineering

• Design and implement evaluation frameworks for LLM outputs, including LLM-as-judge patterns, RAGASstyle retrieval evaluations, regression testing for prompt changes, and output reliability scoring
• Build observability and tracing infrastructure for AI systems: cost monitoring, latency tracking, token usage, failure analysis, and audit trails for BSP compliance
• Establish prompt versioning, testing, and governance practices to ensure reproducibility and auditability of AI system behavior

Technical Leadership & Guidance

• Serve as the team's technical authority on LLM behavior, prompt engineering best practices, and agentic system design patterns

• Drive technical improvements within the team, co-mentor developers, and ensure alignment with best practices in AI engineering.
• Champion AI fluency and proficiency across the team, ensuring a shared understanding of core AI/LLM concepts, capabilities, limitations, and costs to support sound technical and business decisions.

Strategy Implementation

• Translate group-level technical strategies (Agentic AI projects, Enterprise AI MCPs) into actionable implementations across enterprise automation workflows
• Design and maintain Model Context Protocol (MCP) servers and tool schemas that enable reliable AI-toenterprise-system integration
• Promote a Platform-as-a-Service (PaaS) mindset: Building reusable, scalable AI components (prompt templates, evaluation suites, agent blueprints) that the broader Enterprise AI team can leverage

Technology Evaluation

• Conduct technology evaluations for AI tools, frameworks, and models, and recommending the most suitable options with sound cost/performance tradeoffs AI Governance and Compliance
• Ensure all AI systems produce auditable decision trails that satisfy BSP regulatory requirements and internal InfoSec/CISO standards
• Implement guardrails, content filtering, and safety mechanisms for production LLM systems handling sensitive financial data

DISPLAYED SKILL MASTERY

LLM & Agentic AI Expertise

• Deep understanding of LLM behavior in production: Hallucination patterns, context window management, model-specific characteristics, cost/latency tradeoffs across providers (OpenAI, Anthropic, open-source models)
• Proficiency in designing agentic systems such as multi-step orchestration, tool/function calling, memory architectures, and reliable error recovery patterns
• Experience building evaluation frameworks for AI outputs such as automated quality scoring, regression testing, and reliability metrics

Software Engineering

• Strong Python development with emphasis on API design (FastAPI), containerization (Docker), and CI/CD pipelines
• Ability to design and deploy production inference services such as model serving, API gateway patterns, authentication, and rate limiting
• Experience with prompt engineering at systems level: Versioning, A/B testing, governance, and documentation
• Proficiency in Git for version control, including branching strategies, code reviews, and collaborative development workflows.

Software Development Leadership and Mentorship

• Drive team-wide technical improvements and mentor team members on both software engineering/AI ML systems/technologies.
• Drive team-wide adoption of AI engineering standards including evaluation practices, prompt governance, and production reliability patterns

Strategic Thinking

• Translate high-level strategies into practical implementations that leverage both LLMs and enterprise automation tools.

Behavioral Skills

• Ability to functionally decompose complex problems into simple, straightforward solutions
• Have a complete understanding of the various application/ system interdependencies and limitations
• Ability to operate and innovate in a lean team with a fast-paced environment, balancing both strategic and tactical needs
• Detailed-oriented and the ability to spot and fix errors in complex code
• Analytical and critical problem-solving abilities
• Ability to perform tasks independently
• Good presentation and report writing skills

REQUIRED QUALIFICATIONS

• Education: Bachelor’s Degree in Computer Science, Engineering, Information Technology, or related field required. Master’s Degree holders in Computer Science, Artificial Intelligence, or related fields preferred.
• Experience: 5+ years in software engineering or AI/ML development, with at least 2 years of hands-on experience building production systems using large language models (LLMs). Demonstrated experience designing agentic AI systems, building LLM evaluation frameworks, and deploying AI services in containerized environments. Experience in regulated industries (banking, finance, insurance) is strongly preferred.
• Technical Skills: Expert-level knowledge of Python and API development. Deep understanding of LLM behavior, prompt engineering, and agentic patterns. Familiarity with evaluation frameworks (RAGAS, LLMas-judge, custom metrics). Experience with agent orchestration tools (e.g., n8n, Dify, Hermes), cloud platforms (e.g. Azure, AWS), and Low-Code/No-Code tools is a plus, not a prerequisite.
• Leadership: Proven ability to serve as a technical authority on AI/LLM systems, guide team members effectively, and contribute directly to development tasks involving agentic AI and enterprise platforms.
• Industry Knowledge: Deep understanding of enterprise AI applications, agentic AI architectures, Model Context Protocol (MCP), and emerging trends in LLM/GenAI-powered automation. Understanding of BSP regulatory requirements and financial services compliance frameworks is a plus.