{"id":2903403,"url":"https://alion.io/job/harmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering","title":"DevOps Lead (GCP, AWS) – Data Engineering","company":{"id":3810666,"name":"Harmony Data Integration Technologies","domain":"harmonydi.com","url":"https://alion.io/company/harmony-data-integration-technologies","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"WP Job Openings","truth_index":{"grade":"B","score":75,"open_postings":4,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-28T05:59:10Z"}},"role":"DevOps","role_family":"DevOps","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Kinesis","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Apache Kafka","optional":false},{"name":"ArgoCD","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"dbt","optional":false},{"name":"Docker","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"GitHub Actions","optional":false},{"name":"GitLab CI","optional":false},{"name":"Google BigQuery","optional":false},{"name":"Helm","optional":false},{"name":"IAM","optional":false},{"name":"Jenkins","optional":false},{"name":"Kubernetes","optional":false},{"name":"Least Privilege","optional":false},{"name":"Platform Engineering","optional":false},{"name":"Python","optional":false},{"name":"Shift-Left Security","optional":false},{"name":"Snowflake","optional":false},{"name":"SonarQube","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Terraform","optional":false},{"name":"Vault","optional":false},{"name":"Apache Iceberg","optional":true},{"name":"CIS Benchmarks","optional":true},{"name":"Delta Lake","optional":true},{"name":"Dimensional Modeling","optional":true},{"name":"Feature Store","optional":true},{"name":"FinOps","optional":true},{"name":"Grafana","optional":true},{"name":"Great Expectations","optional":true},{"name":"ISO 27001","optional":true},{"name":"Loki","optional":true},{"name":"MLFlow","optional":true},{"name":"Prometheus","optional":true},{"name":"Vertex AI","optional":true}],"status":"live","first_seen_at":"2026-04-13T10:36:03Z","employer_posted_date":"2026-08-19","last_verified_at":"2026-10-28T10:54:05Z","board_verified":true,"closed_at":null,"days_open":198,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":197},"description":"We are looking for a highly skilled and motivated DevOps Lead to design, build, and operate scalable, secure, and highly available cloud and data platforms, primarily on Google Cloud Platform (GCP) with Amazon Web Services (AWS) as a secondary environment.\nThis is a hybrid platform role. Alongside core DevOps ownership - Terraform, Kubernetes, CI/CD, cloud security, and observability - you will also contribute substantially to our data engineering function: building and operating production Python data pipelines, orchestrating batch and streaming workflows, and owning the infrastructure that sits underneath our data platform.\nThe ideal candidate has spent several years building data pipelines as a data engineer and has since moved into a DevOps / SRE / Platform Engineering role, or has run platform engineering for data-intensive products. We are looking for someone who can credibly wear both hats: infrastructure and automation first, data engineering a close second.\nDevOps & Cloud Infrastructure Responsibilities:\nDesign, implement, and manage cloud infrastructure across GCP and AWS using Terraform and other Infrastructure-as-Code practices.\nDeploy and maintain Kubernetes clusters and manage application workloads using Helm and/or ArgoCD.\nBuild and enhance secure, scalable CI/CD pipelines using GitHub Actions, Jenkins, GitLab CI, and ArgoCD.\nImplement best practices in containerization using Docker, including image hardening and container runtime security.\nIntegrate quality gates in CI/CD, including automated testing, code coverage, and SonarQube configuration.\nEnsure secure CI/CD processes, including secrets management, least-privilege access, and hardened build agents.\nImplement and maintain cloud security best practices: IAM roles and policies, network segmentation, encryption, and vulnerability management.\nCollaborate with security teams to embed DevSecOps practices (shift-left security, scanning in pipelines, policy-as-code).\nContribute to system architecture, monitoring, and observability using modern tools and practices.\nParticipate in incident response, post-incident reviews, and root cause analysis.\nData Engineering & Data Platform Responsibilities:\nDesign, build, and maintain production-grade ETL/ELT pipelines in Python for both batch and streaming workloads.\nBuild and operate workflow orchestration using Airflow / Cloud Composer, including scheduling, dependency management, retries, and alerting.\nDevelop and optimise data models and transformations in BigQuery (and equivalents such as Snowflake or Redshift), with attention to partitioning, clustering, and query cost.\nOwn the infrastructure and deployment lifecycle of data services - provisioning, autoscaling, upgrades, and cost management.\nImplement data quality, validation, lineage, and observability checks so pipeline failures and data drift are caught early.\nApply software engineering discipline to data workloads: version control, code review, automated testing, and CI/CD for pipelines and transformations.\nWork with analytics, ML, and product teams to make data reliably available, secure, and well governed.\nMust have skills:\n-8+ years of total professional experience in software, data, or infrastructure engineering.\n-Minimum 4 years in a dedicated DevOps / SRE / Platform Engineering role.\n-Substantial prior hands-on data engineering experience at a senior level (pipeline development, data warehousing, and orchestration).\n-Strong proficiency in Google Cloud Platform (GCP) and solid working experience with AWS.\n-Hands-on experience with Terraform and Infrastructure as Code (IaC).\n-Strong experience with Kubernetes, Docker, and Helm in production.\n-Expertise with CI/CD tools: Jenkins, GitLab CI, GitHub Actions, ArgoCD.\n-Strong production-grade Python - not scripting only, but maintainable pipeline and tooling code - plus solid Bash.\n-Advanced SQL and hands-on experience with a cloud data warehouse (BigQuery preferred; Snowflake or Redshift also considered).\n-Experience with workflow orchestration (Airflow / Cloud Composer) and a distributed processing framework (Dataflow/Apache Beam, Spark, or dbt).\n-Experience with streaming or event-driven data (Pub/Sub, Kafka, or Kinesis).\n-Experience with SonarQube and static code analysis integration.\n-Practical cloud security experience (IAM, VPC, firewalls/security groups).\n-Secrets management experience (Secret Manager / Secrets Manager / Vault).\n-Experience with image and dependency scanning.\n-Knowledge of secure software development practices.\n-Excellent written and verbal communication skills.\nGood to Have:\n-Certifications: GCP Professional DevOps Engineer, GCP Professional Data Engineer, GCP Architect, AWS DevOps Engineer, etc.\n-Experience with dbt and modern analytics engineering practices.\n-Data modelling experience (dimensional modelling, lakehouse formats such as Iceberg or Delta Lake).\n-Data quality and testing frameworks (Great Expectations, Soda, dbt tests).\n-Exposure to ML platform support (Vertex AI, MLflow, feature stores, model deployment pipelines).\n-Knowledge of compliance frameworks (ISO 27001, CIS Benchmarks, NIST) and data governance / PII handling.\n-Experience with monitoring tools (Prometheus, Grafana).\n-Experience with logging stacks (ELK, Loki, etc.).\n-Exposure to policy-as-code and security tools (OPA, scanners).\n-Cloud cost optimisation / FinOps experience, particularly for data workloads.\n-Scripting or service development in Go.","description_format":"text","description_chars":5458,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-10-20T07:19:04Z"}],"visa":[],"liveness":{"score":7,"band":"cold","label":"Long shot","p_open":1,"p_active":0.25,"p_room":0.28,"age_days":197,"expected_fill_days":27,"reasons":["conf:4","velocity","win:tail","crowd:"],"computed_at":"2026-10-28T05:59:10Z"},"pay":null,"html_url":"https://alion.io/job/harmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering","json_url":"https://alion.io/job/harmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering.json","meta":{"generated_at":"2026-10-28T12:07:44Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":15965,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-29T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3810666},"rest":"https://alion.io/mcp/rest/get_company?id=3810666"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fharmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fharmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fharmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/harmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fharmony-data-integration-technologies-devops-lead-gcp-aws-data-engineering"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2903403}},{"tool":"search_jobs","arguments":{"company":"harmonydi.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2903403},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}