Specialist I - DevOps Engineering

Job Title: Senior AWS / DevOps Engineer (ML & Data Automation)

Position Overview

We are seeking a highly skilled Senior AWS / DevOps Engineer to design, build, and maintain secure, scalable cloud infrastructure supporting our AI/ML initiatives and automated data extraction pipelines. In this role, you will bridge the gap between cloud infrastructure, data engineering, and machine learning, ensuring that our Large Language Model (LLM) modules and automated scraping frameworks run seamlessly within a highly secure, isolated network.

The ideal candidate will have extensive experience with AWS networking and serverless orchestration, alongside hybrid-cloud exposure integrating with Azure Active Directory (AD) and enterprise CI/CD pipelines.

Key Responsibilities

• Infrastructure & Networking: Design and maintain a secure AWS environment within a Factory Account structure. Configure and manage Factory VPCs, Private Subnets, VPC Endpoints, and S3 Gateway Endpoints to ensure absolute network isolation and zero public internet exposure for backend data.
• Data Pipeline Orchestration: Build, optimize, and maintain event-driven data extraction pipelines utilizing AWS Step Functions and Lambda to orchestrate distributed Scrapy (Python) web-scraping workloads.
• Application Containerization & Deployment: Manage containerized applications using Amazon ECR. Support and maintain the hosting infrastructure for frontend Streamlit applications running within the AWS environment.
• Hybrid Cloud Integration: Collaborate with cross-functional teams to implement secure Single Sign-On (SSO) and user authentication utilizing Merck Azure AD. Maintain and optimize Azure Deployment Pipelines for robust CI/CD across environments.
• Security & Database Management: Manage relational databases via Amazon RDS and secure object storage using S3. Enforce strong security postures by centralizing credentials, API keys, and connection strings within AWS Secrets Manager.
• Monitoring & Observability: Implement comprehensive logging, tracking, and ing metrics across all containerized workloads, serverless functions, and network paths using Amazon CloudWatch.

Required Technical Skills

1. Amazon Web Services (AWS)

• Networking: Advanced knowledge of VPC architecture, Private Subnets, VPC Endpoints (PrivateLink), and S3 Gateway Endpoints.
• Compute & Orchestration: Deep expertise in AWS Lambda (Serverless) and AWS Step Functions for workflow state machine management.
• Storage & Databases: Hands-on experience with Amazon S3 and Amazon RDS administration.
• Security & Management: Proficient with AWS Secrets Manager, EventBridge (CloudWatch Events), and Amazon ECR.
• Observability: Strong experience setting up dashboards and s in Amazon CloudWatch.

2. Azure & Hybrid Integration

• Identity Management: Enterprise-level experience integrating applications with Azure AD for authentication/SSO.
• DevOps: Proven experience building and managing CI/CD automated workflows via Azure Deployment Pipelines.

3. Data Engineering & AI/ML Development

• Data Extraction: Strong familiarity with Python-based data extraction, specifically the Scrapy framework.
• AI/ML Concepts: Foundational understanding of hosting, scaling, and managing infrastructure for Machine Learning (ML) models and LLM Modules.
• Frontend UI: Experience deploying and supporting Streamlit applications.