{"id":4549555,"url":"https://alion.io/job/distributed-spectrum-technical-lead-data-engineering","title":"Technical Lead, Data Engineering","company":{"id":673687,"name":"Distributed Spectrum │","domain":"distributedspectrum.com","url":"https://alion.io/company/distributedspectrum","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":75,"open_postings":3,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-18T05:48:20Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["New York, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":160000,"max":200000,"currency":"USD","period":"year","gross":null,"usd_annual":200000},"salary_estimate":null,"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Apache Hudi","optional":false},{"name":"Apache Iceberg","optional":false},{"name":"AWS","optional":false},{"name":"AWS Step Functions","optional":false},{"name":"Dagster","optional":false},{"name":"Delta Lake","optional":false},{"name":"DuckDB","optional":false},{"name":"FAISS","optional":false},{"name":"Feature Store","optional":false},{"name":"Flink","optional":false},{"name":"LanceDB","optional":false},{"name":"Milvus","optional":false},{"name":"pgvector","optional":false},{"name":"Pinecone","optional":false},{"name":"Prefect","optional":false},{"name":"Python","optional":false},{"name":"Qdrant","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Trino","optional":false},{"name":"Weaviate","optional":false},{"name":"Amazon Kinesis","optional":true},{"name":"Apache Kafka","optional":true},{"name":"Apache Pulsar","optional":true},{"name":"C++","optional":true},{"name":"PostgreSQL","optional":true},{"name":"Rust","optional":true},{"name":"Time Series","optional":true}],"status":"live","first_seen_at":"2026-11-18T17:12:59Z","employer_posted_date":"2026-11-18","last_verified_at":"2026-11-18T18:20:53Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"DS creates systems that power the next generation of radio spectrum intelligence. We collect radio data from all over the world, train neural networks to decipher it, and run them on the smallest chips we can. We’re solving a new, technically hard problem where nothing from other fields works out of the box, and along the way, we’ve built our own stack from scratch, including entirely new embedding model architectures, custom GPU kernels, and much more.\nJoining DS means owning major parts of a fast-growing AI research organization, joining a collaborative, talent-dense team with decades of experience in probabilistic ML, accelerated computing, embedded systems, and signal theory, and growing your career in the areas that interest you. You’ll fit in if you want to come to work for the problem itself and don’t want to choose between technical rigor, business value, and real-world impact.\nWe work with high ownership and trust.\nThe Role\nDS collects radio data at a scale few organizations ever see - continuous, high-rate streams from sensors around the globe. We're hiring a Technical Lead, Data Engineering to design the platform that turns that firehose into an asset: a data lakehouse that serves researchers training models, production systems running inference, and agents retrieving context in real time.\nYou'll own the architecture from ingestion through storage, cataloging, vector search, and access, and you'll design it explicitly for AI/ML workloads, not just analytics.\nWhat You'll Do\nArchitect and build our data lakehouse: ingestion pipelines, open table formats, partitioning and compaction strategies, cataloging, and governance\n\nDesign and operate vector database infrastructure for embedding storage, similarity search, and retrieval at scale - choosing, tuning, and evolving the right systems for our workloads\n\nBuild large-scale data management systems: lifecycle and retention, lineage, versioning of datasets for reproducible training, quality monitoring, and cost management across petabyte-class storage\n\nDesign data platform capabilities that directly support ML use cases - feature and embedding pipelines, training-set assembly, evaluation datasets, and low-latency retrieval for agents\n\nEstablish schema and data-contract standards across teams, and build tooling that lets researchers and engineers self-serve\n\nOwn reliability and performance of data pipelines in production, including observability and failure handling\n\nMentor engineers and lead technical design across the data domain\n\nWhat We're Looking For\n4-6+ years of software / data engineering experience, including several years designing and operating large-scale data platforms in production\n\nDeep experience with lakehouse architectures and technologies - e.g., Apache Iceberg, Delta Lake, or Hudi; Parquet; Spark, Flink, Trino, DuckDB, or similar engines\n\nHands-on experience with vector databases and embedding retrieval (e.g., pgvector, Milvus, Qdrant, Weaviate, Pinecone, LanceDB, or FAISS-based systems), including indexing trade-offs and scaling\n\nStrong experience with AWS or another major cloud provider - object storage, managed data services, compute, and cost optimization at scale\n\nFluency in Python and SQL; experience with orchestration tools (Airflow, Dagster, Prefect, Step Functions, etc.)\n\nDemonstrated experience designing data architectures specifically for ML / AI workloads: training data pipelines, feature stores, dataset versioning, or retrieval systems\n\nStrong grasp of data modeling, consistency, and the trade-offs between batch and streaming\n\nMust currently hold or be eligible to obtain and maintain a U.S. government security clearance\n\nNice To Have\nExperience with streaming ingestion at high volume (Kafka, Kinesis, Pulsar)\n\nExperience with time-series, geospatial, or signal / sensor data\n\nFamiliarity with data governance and security requirements in regulated environments\n\nExperience with Rust, Go, or C++ for performance-critical data paths\n\nWho Thrives at Distributed Spectrum\nFast learners over specific backgrounds - We care more about how quickly you can pick up new skills than where you’ve worked before.\n\nIntellectual honesty - The right answer matters more than being right. You challenge assumptions, test ideas, and pivot when needed.\n\nAdaptability - We’re organized, but sometimes things change quickly. You find a way to make it work and balance short-term deliverables with long-term goals.\n\nOwnership of outcomes - You optimize your own time, focus on what matters to deliver quickly, and cut out inefficiencies.\n\nNot building in a vacuum - You stay connected to the rest of our teams and our customers to make sure all the pieces fit together.\n\nWhat We Offer\nAbove-market salary, equity, and benefits package.\n\nEarly Series A Equity\n\nExcellent health, dental, and vision coverage\n\n401(k) match - up to 4% of your salary\n\nFlexible PTO\n\nDaily office lunches in NYC\n\nITAR Requirements\nTo conform to U.S. Government technology export regulations, including the International Traffic in Arms Regulations (ITAR) you must be a U.S. citizen, lawful permanent resident of the U.S., protected individual as defined by 8 U.S.C. 1324b(a)(3), or eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here.","description_format":"text","description_chars":5304,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":true,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Sensors & Transducers","RF & Wireless Hardware"],"lifecycle":[{"event":"open","at":"2026-11-18T18:20:53Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":30,"reasons":["conf:2","win:early"],"computed_at":"2026-11-18T21:12:42Z"},"pay":{"stated_usd_annual":200000,"is_top_pay":true},"html_url":"https://alion.io/job/distributed-spectrum-technical-lead-data-engineering","json_url":"https://alion.io/job/distributed-spectrum-technical-lead-data-engineering.json","meta":{"generated_at":"2026-11-18T21:12:42Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":720,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":673687},"rest":"https://alion.io/mcp/rest/get_company?id=673687"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fdistributed-spectrum-technical-lead-data-engineering"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fdistributed-spectrum-technical-lead-data-engineering"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fdistributed-spectrum-technical-lead-data-engineering"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/distributed-spectrum-technical-lead-data-engineering\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fdistributed-spectrum-technical-lead-data-engineering"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4549555}},{"tool":"search_jobs","arguments":{"company":"distributedspectrum.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4549555},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}