Staff Engineer - Core Knowledge Platform

Responsibilities:

• Lead architecture and implementation of the knowledge_base stackInfoNode models, persistent graph representations, semantic index, Cypher DSL, and storage adapters using immutable data models, declarative transformations, and interpreter-style flows.
• Steward the metaclass/pydantic infrastructure: keep the InfoNode registry coherent, codify field annotations (semantic, edge, key roles), and make the schema-definition DSL approachable to other engineers.
• Design and harden ingestion pipelines (schema, stats, query history, BI model inference, column indexing) as deterministic interpreters over warehouse state; ensure they are composable, observable, and easy to extend.
• Advance the semantic index by defining embedding schemas, normalisation strategies, and retrieval DSLs that keep concept search consistent as we evolve vector backends and storage layers.
• Build meta-programming utilities, code generation, InfoNode introspection, and query builders that let other teams extend the graph and DSL safely without touching low-level plumbing.
• Drive reliability/observability for ingestion runs, graph migrations, and index updates; implement diagnostics, rollback strategies, and property-based tests for functional pipelines.
• Partner with interpretation/conversation teams to expose new graph/search capabilities via formal DSL contracts and query APIs, and mentor engineers on FP thinking, DSL design, and testing strategies.
• Improve developer experience: local fixtures, REPL-like tooling for graph queries, schema compilers for InfoNodes, and CI coverage emphasising deterministic behaviour.

Requirements:

• 8+ years building large-scale data or knowledge-graph platforms; expert-level Python or equivalent FP language experience that translates into Pythonic FP practices.
• Demonstrated mastery of functional programming, persistent data structures, and metaprogramming (immutable models, algebraic data types, interpreter pipelines, DSL compilers).
• Experience designing metaclass- or macro-driven schema systems (e. g., custom Pydantic bases, static analysers, IR builders) and making them usable across a codebase.
• Hands-on experience with graph databases/Cypher-like DSLs and ingestion/ELT systems spanning warehouses (Snowflake, Redshift, BigQuery, etc. ) and semantic/ML consumers.
• Practical knowledge of vector search systems (Milvus, Lance, FAISS, pgvector, etc. ) and embedding lifecycle management.
• Product-focused infrastructure mindset: stable APIs, thoughtful abstractions, strong debugging instincts, and a balance of rapid iteration with architectural rigour.