Senior Software Engineer (Platforms)

We're seeking a Senior Software Engineer who excels at building high-performance distributed systems and thrives in a fast-paced startup environment. You'll be working on cutting-edge data infrastructure challenges that directly impact how Fortune 500 industrial companies manage their most critical operational data.

Responsibilities:

• Platform Ownership: Design, build, and operate the core serverless execution engine and workflow orchestration layer that serve as foundational primitives for CDF's AI and automation capabilities.
• Reliability Engineering: Own uptime, latency SLOs, and incident response for platform services, ensuring functions execute deterministically and workflows progress without data loss or silent failures.
• Scalability: Architect for multi-tenant and multi-cloud, high-throughput workloads. Design scheduling, queueing, and retry mechanisms that degrade gracefully under pressure.
• API Design: Define and evolve clean API-first architecture, versioned REST, and event-driven APIs that downstream engineering teams and external customers depend on.
• Observability: Instrument services with distributed tracing, structured logging, and alerting (OpenTelemetry / Prometheus / Grafana / Honeycomb stack) so failures surface before customers notice.
• CI/CD and Testing: Champion test automation, unit, integration, and smoke tests and maintain deployment pipelines that ship to production with confidence.
• Performance: Profile and resolve bottlenecks in execution throughput, cold-start latencies, and cross-service call chains driving a snappy platform experience for industrial workloads.
• Cost Efficiency (Bonus): Model compute and storage costs for function execution; identify and implement optimisations that reduce cloud spend without sacrificing reliability.

Requirements:

• 6+ years of Engineering: Proven track record building and operating production backend services at scale.
• Expertise: Deep mastery of JVM languages (Kotlin preferred, Java acceptable), Python (FastAPI), distributed systems patterns, and cloud-native service design (Kubernetes, Azure, GCP, AWS, Private cloud).
• Workflow and Orchestration: Hands-on experience with workflow engines (Conductor, Apache Airflow, or equivalent) and event-driven architectures (Kafka, Pub/Sub).
• Data and Storage: Comfortable working with relational databases (PostgreSQL) and non-relational databases, object storage (Data Lakes), and caching layers (Redis) in multi-tenant environments.
• Observability Stack: Practical experience with OpenTelemetry, Prometheus, and Grafana for instrumentation and operational insight.
• ML Platform Exposure: Experience supporting ML workloads and notebooks in production, whether through job scheduling, resource management, experiment tracking integration, or model serving infrastructure.
• Contextualization Domain (Bonus): Familiarity with industrial knowledge graph construction, entity resolution, or NLP/CV pipelines as they relate to industrial asset data is a strong differentiator.
• Full-Stack Awareness (Bonus): Familiarity with React or TypeScript is a plus for consuming and dogfooding your own platform's developer tooling.
• The Platform Thinking Spirit: A passion for building composable, well-documented, and automated platform systems that empower other engineers, including ML engineers, to build faster.

Good to have:

• ML Workload Support: Build and extend platform primitives for compute scheduling, environment management, and secrets handling that enable ML engineers to run model training, fine-tuning, and batch inference jobs reliably.
• Contextualization Pipelines: Support the engineering infrastructure behind Cognite's contextualization capabilities (entity matching, asset hierarchy inference, and P& ID parsing) by ensuring the platform can orchestrate long-running, GPU-aware, and data-intensive ML workflows without manual intervention.
• Vector and Embedding Infrastructure (Bonus): Familiarity with serving or storing vector embeddings to support semantic search and RAG-based contextualization use cases.
• Model Lifecycle Awareness: Understand model versioning, A/B experiment tracking, and the boundary between platform concerns and ML framework concerns, so the platform stays lean while ML teams stay unblocked.