{"id":37443,"url":"https://alion.io/job/lightfield-software-engineer-data-infrastructure-staff","title":"Staff Software Engineer, Data Infrastructure","company":{"id":8339,"name":"Lightfield","domain":"lightfield.com","url":"https://alion.io/company/lightfield","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":75,"open_postings":5,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":48,"computed_at":"2026-10-17T05:46:00Z"}},"role":"Backend","role_family":"Backend","seniority":"staff","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States","Cambridge, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":180000,"max":300000,"currency":"USD","period":"year","gross":null,"usd_annual":300000},"salary_estimate":null,"experience_years_min":7,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Apache Kafka","optional":false},{"name":"BullMQ","optional":false},{"name":"ClickHouse","optional":false},{"name":"Flink","optional":false},{"name":"Llama","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Redis","optional":false},{"name":"Typesense","optional":false},{"name":"World Models","optional":false},{"name":"JavaScript","optional":true},{"name":"Node JS","optional":true}],"status":"live","first_seen_at":"2026-08-03T20:57:20Z","employer_posted_date":"2026-08-03","last_verified_at":"2026-10-17T10:27:47Z","board_verified":true,"closed_at":null,"days_open":74,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":74},"description":"About Lightfield\nLightfield is reimagining CRM as a world model of a business. By learning from emails, meetings, and customer conversations, we’re building a living understanding of how a company works-so AI can help teams anticipate what comes next, make better decisions, and take action.\nMore than 5,000 companies have used the product since its November launch. We’ve raised more than $70M from Andreessen Horowitz, Coatue, Maverick Ventures, and Greylock to attack the biggest software category in the world. We previously built Tome, which reached more than 25 million users, and our team brings experience from Llama, Instagram, Google, and Salesforce.\nAbout the role\nWe're building the infrastructure foundation for a fast-growing AI product company serving thousands of customers:\nOur Postgres fleet serves 5B+ queries a month - roughly 2,500 QPS steady state, with sustained spikes past 25,000 QPS - and database workload more than doubled last month.\n\nRedis sustains ~50,000 commands per second behind a job platform that executes 10M+ background job runs a day across ~170 queues.\n\nWe ingest tens of millions of emails and calendar events a month.\n\nThat growth creates scaling pressure across backend systems, infrastructure, and data infrastructure. We're hiring a staff-level engineer who spikes in data infrastructure but is excited to work across backend systems, infrastructure, and product-facing data problems. The work is close to the product, close to customers, and close to production.\nAs a Software Engineer, Data Infrastructure, you'll build the next generation of our data systems. We got remarkably far on a deliberately simple stack: Postgres as the system of record, a sharded transactional outbox for change events, Redis-buffered sync into Typesense for search, BullMQ for processing, and Postgres-backed customer-facing analytics with per-organization row-level security.\nThe next phase is evolving that pragmatic foundation into best-practice data architecture: change data capture, event modeling, schema design, query performance, freshness guarantees, and the right boundary between transactional and analytical workloads.\nThe system of record itself is unusual. Customers define their own objects, attributes, and relationships at runtime, so the core data model is a schema-flexible, graph-shaped store: entity-attribute-value with typed edges, versioned attribute values, and relationship history. That makes schema design, indexing, and query performance genuinely hard problems rather than routine tuning.\nThe surface area is wider than analytics: customer-facing dashboards, historical and audit data, datasets that power pipeline-generation products, and evaluation data that measures our AI agents. This is data infrastructure work, not a BI or dashboarding role. It's a good fit for someone who likes high-volume data systems, pragmatic architecture decisions, and building foundations that product and engineering teams can actually depend on.\nThis role can be based in San Francisco or Cambridge. In San Francisco, you'd work from our HQ alongside the founders and most of the engineering team. In Cambridge, you'd join an initial group of staff-level engineers at our new, infrastructure-focused Kendall Square site, working alongside one of our most senior infrastructure engineers. We aim to build the site and organization around this group as the company scales.\nWhat you'll do\nScale the analytics engine behind customer-facing dashboards, tackling query performance under row-level security, workload isolation, read architecture, and observability as data volume grows.\n\nDesign the ingestion paths, event models, schemas, and query patterns that move data from transactional writes into search, dashboards, and history-with clear guarantees around freshness, correctness, replay, and failure recovery.\n\nEvolve our schema-flexible, graph-shaped data model so customer-defined objects, attributes, and relationships remain fast to query as their size and complexity grow.\n\nBuild the foundations for historical reporting and auditability, including attribute versioning, relationship history, and change capture.\n\nBuild reliable data systems for usage metering, pipeline generation, and AI evaluation, where errors have direct customer, product, or financial consequences.\n\nDecide when our existing architecture remains the right foundation and when new analytical, streaming, or workflow systems earn their added complexity.\n\nSet the technical direction, abstractions, ownership boundaries, and engineering practices for data systems as the company grows.\n\nWhat your first year looks like\nScaling the analytics serving path behind customer-facing dashboards is the anchor project, but the work stays close to the product. The current slate also includes:\nZero-downtime schema migrations for an 18-collection Typesense search deployment.\n\nA usage-metering pipeline for consumption billing.\n\nHistorical and audit data modeling.\n\nEvaluation data infrastructure for our AI agents.\n\nExpect a first year that mixes foundational data systems with product-shaped projects, with the scope to own the technical direction for how data is modeled, moved, and served across Lightfield-and to shape the architecture, abstractions, and team we build as the company scales.\nWhat we're looking for\n7+ years experience.\n\nStrong software engineering fundamentals.\n\nExperience owning production data systems where query plans, replication lag, backfills, data freshness, schema evolution, or data correctness had real user-facing consequences.\n\nComfort debugging across multiple layers of the stack.\n\nGood judgment about when to make a tactical fix and when to invest in a more durable platform or architecture change.\n\nProduct orientation: you care about how data infrastructure decisions affect customers, users, and engineering velocity.\n\nClear communication, strong ownership, and a bias toward practical tradeoffs.\n\nHelpful experience\nYou do not need all of these:\nClickHouse, OLAP systems, event pipelines, data warehouses, or analytical infrastructure.\n\nKafka, Flink, Spark, Iceberg, or similar streaming and lakehouse systems.\n\nPostgres at scale, and the boundary between OLTP and OLAP systems.\n\nAPIs, queues, workflow systems, and distributed systems.\n\nObservability, incident response, service ownership, and production debugging.\n\nData for ML/AI systems: enrichment pipelines, eval harnesses, or data-quality tooling.\n\nExperience in a high-growth product environment.\n\nWhy this role is interesting\nYou'd be building our analytical data architecture from close to the beginning - the foundations are deliberately simple, and the architecture that scales them is yours to shape. Customer-facing data products are on the roadmap, database workload more than doubled last month, and the foundations you build will carry the company for years.\nBenefits & Perks\nCompetitive salary\n\nMeaningful early equity\n\nHealth insurance (medical, dental, vision)\n\n3 weeks of PTO\n\n11 paid company holidays + we enjoy a winter holiday break\n\n3 months of paid family leave\n\nWednesdays work from home\n\nRegular team dinners, events, offsites, and retreats\n\n401k plan\n\nOther perks include: commuter and lunch stipend","description_format":"text","description_chars":7235,"description_truncated":false,"requirements":{"experience_years_min":7,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["401k plan","Equity","Health insurance","Parental leave"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["CRM Software","Marketing & Sales AI"],"lifecycle":[{"event":"open","at":"2026-08-03T20:57:20Z"}],"visa":[],"liveness":{"score":24,"band":"cold","label":"Long shot","p_open":1,"p_active":0.664,"p_room":0.36,"age_days":74,"expected_fill_days":48,"reasons":["conf:2","velocity","win:tail","crowd:"],"computed_at":"2026-10-17T05:46:00Z"},"pay":{"stated_usd_annual":300000,"is_top_pay":true},"html_url":"https://alion.io/job/lightfield-software-engineer-data-infrastructure-staff","json_url":"https://alion.io/job/lightfield-software-engineer-data-infrastructure-staff.json","meta":{"generated_at":"2026-10-17T12:35:23Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16698,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":8339},"rest":"https://alion.io/mcp/rest/get_company?id=8339"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Flightfield-software-engineer-data-infrastructure-staff"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Flightfield-software-engineer-data-infrastructure-staff"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Flightfield-software-engineer-data-infrastructure-staff"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/lightfield-software-engineer-data-infrastructure-staff\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Flightfield-software-engineer-data-infrastructure-staff"}]}