Architect Python & Data Infrastructure Engineer - Python+Cloud,9+years,Immediate joiners only

Position: Lead / Architect Python & Data Infrastructure Engineer

Total experience: 9+ yrs

Employment Type: 1 Year Contract

Desired Skills and Experience:

9+ years of experience building and operating production backend services, distributed systems, or large-scale data platforms.

Expertise in Python: Deep proficiency in the modern Python ecosystem (FastAPI, uv, async I/O, typing).

Database & Caching Mastery: Advanced knowledge of PostgreSQL (query optimization, ACID transactions, locking mechanisms, indexing) and Redis (caching strategies, pub/sub, distributed locks).

Container & Cloud Orchestration: Hands-on experience with Kubernetes (k8s), Docker, Helm, and AWS cloud architecture.

Infrastructure-as-Code (IaC): Proven track record with CDKTF (Cloud Development Kit for Terraform) or Terraform.

Workflow Orchestration: Production experience with pipeline orchestrators such as Dagster, Celery, or Apache Airflow.

Observability: Practical experience instrumenting systems with OpenTelemetry, Prometheus, Grafana, or Datadog.

Distributed Systems Fundamentals: Clear understanding of race conditions, deadlocks, consensus primitives, and message-queue semantics.

Good To Have

Experience with Lakehouse architectures (Apache Iceberg, Delta Lake) and CDC (Change Data Capture) via Debezium/Kafka.

Exposure to MLOps workflows (MLflow, Feature Stores, embedding pipelines).

Familiarity with consensus protocols and distributed coordination systems (Raft, Paxos, ZooKeeper, etcd).

Frontend familiarity with React, TypeScript, or Tailwind CSS.

Active contributions to open-source software.

Roles and Key Responsibilities:

1. Backend & Microservices Engineering:

Architect, build, and deploy low-latency, scalable APIs and microservices using modern Python (FastAPI, FastMCP, uv).

Integrate foundational AI/LLM models and external services (e.g., AWS Bedrock).

Define resilient service interfaces, API schemas, and data contracts between internal engineering teams.

2. Distributed Systems & Concurrency Management:

Design robust state-management patterns to handle high-concurrency race conditions across worker pools, message queues, and database rows.

Implement distributed synchronization mechanisms (e.g., distributed locks/mutexes, idempotency keys, and optimistic/pessimistic locking strategies).

Ensure reliable execution, deduplication, and zero-data-loss guarantees across asynchronous task workers.

3. Data Infrastructure & Orchestration:

Develop and maintain scalable job orchestration workflows for streaming and batch processing using Celery, Dagster, or Apache Airflow.

Design, optimize, and maintain relational datastores (PostgreSQL) and in-memory caches (Redis).

Build modern analytical pipelines and data storage layers leveraging Lakehouse architectures (Apache Iceberg, Delta Lake) and Change Data Capture (CDC) patterns.

Enforce end-to-end data quality, governance, schema evolution, and lineage across platforms.

4. Cloud Infrastructure, Kubernetes & CI/CD

Own, automate, and maintain cloud environments on AWS using Infrastructure-as-Code (CDKTF / Terraform).

Containerize services with Docker and manage deployments, auto-scaling (HPA), and load-balancing on Kubernetes (k8s) using Helm.

Build, maintain, and optimize secure GitHub Actions CI/CD deployment pipelines.

5. Observability & Reliability Engineering

Ensure production uptime, high availability, and performance SLA/SLO benchmarks.

Build deep observability, tracing, and alerting stacks across all microservices and data pipelines using OpenTelemetry, Prometheus, Grafana, and Datadog.

Note: Please send Cv only, who can attend the interview weekdays, with short notice period (max 15 days only)

Job Location: Kochi, (work from office only) (Face to Face Interview only)

Employment Type: 1 Year Contract