Software Architect

Description

Grab your cape, become a SuperHyro!

Hyro, the leading Agentic Care Communications platform, enables health systems to safely automate workflows and conversations across their most valuable platforms, services, and channels-including call centers, websites, SMS, mobile apps, and more. Hyro's customers, which include Baptist Health, Hackensack Meridian Health, Tampa General Hospital, and Inova Health System, benefit from AI agents that are fully HIPAA-compliant, fast to deploy, easy to maintain, and simple to scale-generating better conversations, successful patient outcomes, and revenue-driving insights. This is a hands-on role. You’ll write and ship production code, build the prototypes that prove a design before the teams commit to it, and own critical components end to end.

What Are We Looking For?

We are seeking an experienced, hands-onArchitect to design and build the generative AI systems behind Hyro’s AI agents, which help patients across the US get answers, schedule appointments, refill prescriptions, and more.

In this role, you will shape how Hyro builds AI. You will design LLM-based solutions, data and evaluation pipelines, and low-latency voice experiences, and take them into production at scale. You will mentor senior engineers, elevate the engineering bar, and partner closely with Product and cross-functional teams to turn complex healthcare workflows into reliable, high-accuracy agentic capabilities.

Responsibilities

• Own the end-to-end architecture across teams. Write and review design docs, set technical standards, and make the trade-offs between teams explicit.
• Define the model strategy: small vs. large models, self-hosted vs. API, fine-tuning and distillation, and routing and fallback between providers. Weigh quality, latency, and cost.
• Help prototypes from Product Growth become platform capabilities without a rewrite. Balance shipping fast with building something that lasts.
• Design serving infrastructure for small and large models that meets strict latency and availability targets.
• Shape the anomaly-detection and classification systems that catch failures in production conversations.
• Build solutions across different channels, including voice products with minimal latency, high accuracy, and a great user experience.
• Lead new AI initiatives by designing complex functionality and integrating with third parties.
• Build integrations with other systems that speed up development and deployment.
• Design guardrails, hallucination controls, and explainability for healthcare-grade agents.
• Own critical systems hands-on, shipping production code, prototyping POCs, and setting high review standards for core agent services.

Requirements

• 5+ years of experience as an architect or principal engineer.
• 2+ years of experience building ML or LLM-based applications in production.
• Bachelor’s or master’s degree in computer science, data science, or a related field.
• Hands-on in Python. Node/TypeScript is a plus. You still write production code and prototypes.
• Deep production experience with LLM systems: context engineering, RAG, tool calling, structured outputs, and agent frameworks (LangGraph or similar).
• Proven experience designing evaluation frameworks for non-deterministic systems, and using them to make shipping decisions.
• Experience with real-time, latency-sensitive systems (streaming, WebSockets/ WebRTC, event-driven architectures).
• Comfortable switching between quick 0-to-1 builds and hardening platforms at scale.
• Strong interpersonal, leadership, and communication skills with the ability to thrive in a fast-changing environment.

Bonus Points:

• Knowledge of MLOps.
• Voice AI: STT/TTS, speech-to-speech models, LiveKit or similar.
• Model serving and optimization: vLLM/Triton, quantization, GPU efficiency, fine-tuning small models.
• LLM observability tools (Langfuse or similar).

Benefits & Perks

• Unlimited PTO, paid sick leave
• Company-wide long weekends every quarter
• Hybrid, dog-friendly TLV office
• Work from anywhere for up to 2 weeks per year
• Pension, social benefits & Keren Hishtalmut
• Monthly meal allowance

Equal Opportunities

Hyro is proud to be an equal-opportunity employer. We hire talented people regardless of gender, race, ethnicity, ancestry, age, disability, sexual orientation, gender identity or expression, military or veteran status, cultural background, religious beliefs, or any other characteristic protected by federal, state, or local law.