MLOps/Generative AI Engineer - IT Operations
First seen by Alion on Nov 4, 2026.
About the Role:
Ellkay is looking for an MLOps/GenAIOps Engineer to own the end-to-end machine learning operations lifecycle. This role focuses on automating model training, evaluation, deployment, and monitoring workflows while partnering closely with our Full Stack AI/ML Engineers to ensure seamless production deployment of foundation models and classical ML systems. You'll build the infrastructure that enables rapid, safe iteration on AI/ML solutions that impact patient care.
Essential Duties & Responsibilities:
AWS SageMaker Operations & Orchestration:
- Design and implement SageMaker Pipelines with complex DAG authoring for end-to-end ML workflows including data preprocessing, training, evaluation, and deployment.
- Configure and optimize SageMaker Processing Jobs, Training Jobs (PyTorch/HuggingFace containers), and Managed Spot Instances for cost-effective model training.
- Manage SageMaker Model Registry with versioning, lineage tracking, and approval workflows for model governance.
- Implement sophisticated endpoint deployment strategies including blue/green traffic routing, canary deployments, and A/B testing configurations.
- Configure and maintain SageMaker Model Monitor for continuous data quality monitoring, model quality assessment, bias detection, and feature drift alerting.
GenAI Model Lifecycle Management:
- Build operational frameworks for Amazon Bedrock model deployments including provisioned throughput management, guardrail configuration, and usage monitoring.
- Implement automated evaluation pipelines for foundation model outputs with quality gates and human-in-the-loop review workflows.
- Design prompt versioning systems integrated with model registry for reproducible GenAI deployments.
- Monitor GenAI model performance including latency, cost per inference, hallucination rates, and guardrail trigger metrics.
Model Lifecycle Automation:
- Develop automated retraining trigger systems based on data drift, performance degradation, scheduled intervals, or manual triggers.
- Implement evaluation gates as code with configurable thresholds for metrics (F1, precision, recall, AUC, etc.) before model promotion.
- Build automated model promotion workflows from dev - staging - prod with approval gates and rollback capabilities.
- Design rollback automation with traffic shifting strategies to quickly revert to previous model versions upon performance degradation.
Training Data Engineering:
- Build AWS Glue ETL pipelines for training corpus assembly, transformation, and feature engineering at scale.
- Design dataset versioning systems using S3 + manifest files ensuring reproducibility and lineage tracking across model training runs.
- Establish reproducibility patterns including seed management, environment pinning, and deterministic data splitting.
Container Engineering & Registry Management:
- Build optimized Docker containers for training and inference workloads with multi-stage builds, layer caching, and security scanning.
- Manage Amazon ECR repositories with lifecycle policies, image scanning, and vulnerability remediation workflows.
- Implement bring-your-own-container (BYOC) patterns for SageMaker supporting custom frameworks and dependencies.
- Optimize container startup times and resource utilization for cost-effective inference.
ML Observability & Monitoring:
- Design comprehensive CloudWatch dashboards for model health including accuracy, latency, throughput, error rates, and drift metrics.
- Implement custom CloudWatch metrics for business-specific KPIs (e.g., clinical accuracy per condition, false positive rates for critical alerts).
- Build alerting systems for accuracy regression, data drift, concept drift, and model staleness with appropriate escalation paths.
- Create observability frameworks that integrate with AWS X-Ray for end-to-end request tracing from API call through model inference.
CI/CD for ML Artifacts:
- Design and implement GitHub Actions or AWS CodePipeline workflows for automated ML artifact testing, validation, and deployment.
- Build multi-stage promotion pipelines (dev - staging - prod) with automated testing gates and manual approval checkpoints.
- Implement artifact versioning and lineage tracking for models, datasets, feature transformations, and deployment configurations.
- Create integration testing frameworks for ML APIs including performance benchmarking and regression testing.
Infrastructure & Compliance:
- Collaborate with Full Stack AI/ML Engineers to define infrastructure requirements and translate them into production-grade AWS CDK constructs.
- Implement HIPAA-compliant ML pipelines with appropriate encryption, access controls, audit logging, and data residency requirements.
- Design multi-tenant ML infrastructure with tenant isolation, resource quotas, and cost allocation.
- Ensure reproducibility and auditability of all ML experiments and deployments for regulatory compliance.
Qualifications:
Technical Skills:
- AWS SageMaker Expertise: Deep hands-on experience with SageMaker Pipelines, Processing/Training Jobs, Model Registry, Endpoints, Model Monitor, and Managed Spot training.
- AWS Bedrock & GenAI Operations: Production experience managing Amazon Bedrock deployments including Claude family models, Titan Embeddings v2, and Bedrock Guardrails (PHI/PII detection); expertise in provisioned throughput vs on-demand capacity management, model evaluation automation, prompt versioning and registry integration, inference monitoring (latency, cost, token usage), and guardrail performance tracking; ability to build automated quality gates for foundation model outputs.
- AWS ML Services: Proficiency with AWS Glue ETL, Bedrock, S3, ECR, CloudWatch, and Step Functions.
- Container Technologies: Strong Docker skills including multi-stage builds, optimization techniques, and container orchestration.
- Python Programming: Proficient in Python for pipeline orchestration, data processing, and automation scripts (boto3, sagemaker SDK, pandas/polars).
- CI/CD Tools: Experience with GitHub Actions, AWS CodePipeline, or similar CI/CD platforms for ML workflows.
- Infrastructure as Code: Familiarity with AWS CDK, CloudFormation, or Terraform for ML infrastructure provisioning.
- ML Fundamentals: Solid understanding of ML training concepts, evaluation metrics, hyperparameter tuning, and model validation techniques.
- Observability: Experience building custom metrics, dashboards, and alerting systems for production ML systems.
Experience:
- 8+ years of software/data engineering experience with at least 3 years focused on MLOps or production ML systems.
- Proven track record of building automated ML pipelines and deployment workflows at scale.
- Experience with model monitoring, drift detection, and automated retraining systems.
- Healthcare or regulated industry experience strongly preferred.
- Demonstrated ability to operationalize both classical ML and foundation model systems.
Soft Skills:
- Strong collaboration skills to work closely with Full Stack AI/ML Engineers, data scientists, and platform teams.
- Systems thinking with ability to design reliable, fault-tolerant ML infrastructure.
- Excellent troubleshooting and debugging skills for complex distributed systems.
- Commitment to automation, reproducibility, and engineering excellence.
- Strong documentation skills for runbooks, architecture decisions, and operational procedures.
Preferred Qualifications:
- Experience with Amazon Bedrock operational patterns and GenAI model monitoring.
- Familiarity with PyTorch, HuggingFace Transformers, and deep learning training workflows.
- Knowledge of FHIR, HL7 v2, and healthcare data standards.
- Experience with cost optimization strategies for ML workloads (Spot instances, autoscaling, model optimization).
- Contributions to open-source MLOps tools or frameworks.
- AWS certifications (Machine Learning Specialty, Solutions Architect, or DevOps Engineer).
Skills
MLOps, AWS SageMaker, Python, Generative AI, Machine Learning, CI/CD, AWS Glue, AWS Bedrock, IT Operations, AIOps