Senior Engineer, Agent Harness Engineering (IR-557)

Intellectsoft is a software development company delivering innovative solutions since 2007. We operate across North America, Latin America, the Nordic region, the UK, and Europe. We specialize in industries like Fintech, Healthcare, EdTech, Construction, Hospitality, and more, partnering with startups, mid-sized businesses, and Fortune 500 companies to drive innovation and scalability. Our clients include Jaguar Motors, Universal Pictures, Harley-Davidson, and many more where our teams are making daily impactTogether, our team delivers solutions that make a difference. Learn more at www.intellectsoft.net

Our customer advises global pharmaceutical and life sciences clients across commercial strategy, clinical development, regulatory operations, and medical affairs. An increasing share of this work is powered by AI agents designed to handle complex scientific and commercial datasets. This role is split roughly evenly between internal research (developing reusable agent harness components, benchmark evaluations, and reliability frameworks) and client delivery (architecting, deploying, and scaling production-ready agentic systems).

Requirements

• Professional Experience: 2-3 years of professional software engineering experience, including substantive work building LLM-based or agentic systems (beyond prompt-level experimentation).
• Production Deployment: Proven track record of deploying user-facing AI applications to production, including operational experience with monitoring, versioning, rate limiting, cost control, and incident response.
• Core Software Engineering & Python: Strong Python skills with production-grade test coverage, typing, asyncio, Pydantic, Pytest, packaging, version control, CI/CD, and robust API design.
• LLM & Agent Frameworks: Hands-on experience with LLM APIs, function/tool calling, structured outputs, streaming, and context window management. Experience building or extending orchestration frameworks (such as LangGraph, deepagents, OpenAI Agents SDK, or custom harnesses) with a clear view on framework utility.
• Non-Deterministic Evaluation: Ability to define success criteria, design evaluation datasets, and systematically evaluate non-deterministic AI outputs.
• Cloud & Shipping: Working knowledge of containerization and at least one major cloud provider to ship and maintain services independently.
• Communication & Adaptability: Strong written and verbal communication skills with the ability to represent technical positions credibly and thrive amid shifting project priorities.

Nice to have

• Domain Knowledge: Prior exposure to healthcare, pharma, or biotech data (clinical trial data, real-world evidence, regulatory submissions, pharmacovigilance, or commercial analytics) and standards (CDISC, SNOMED, MedDRA).
• Regulated Environments: Familiarity with GxP, 21 CFR Part 11, HIPAA, GDPR, audit logging, and formal system validation.
• Observability & Protocols: Hands-on experience with LLM tracing tools (LangSmith, Langfuse, Braintrust, OpenTelemetry) or developing and hosting MCP servers.
• Retrieval & Models: Experience with RAG pipelines, vector search (pgvector, Qdrant, Weaviate), fine-tuning, distillation, or model serving (vLLM, Bedrock, Vertex AI).
• DevOps & Data Engineering: Infrastructure as Code (Terraform, GitHub Actions, AWS/Azure) alongside large-scale data processing using Polars, Pandas, DuckDB, and advanced SQL.
• Consulting & Ecosystem Engagement: Prior consulting/client services experience, active use of agentic developer tooling (such as Claude Code), or open-source contributions/publications in agent architectures and evaluations.

Responsibilities

• Harness & Orchestration Architecture: Design and build core agent harnesses, covering function calling, tool interfaces, context assembly, retrieval integration, subagent delegation, retry logic, and failure handling.
• Prototype to Production Delivery: Take agentic systems from initial concept to scalable production, managing deployment, monitoring, cost, latency, and operational support against live enterprise client data.
• Evaluation Infrastructure: Build and maintain rigorous evaluation suites, task benchmarks, scoring rubrics, and regression tests using tools like Langfuse, Pydantic Logfire, or custom SDKs to drive architectural decisions.
• System & Tool Integration: Integrate client enterprise systems, data sources, and internal APIs as accessible agent tools, including building and managing Model Context Protocol (MCP) servers.
• Solution Consulting & Requirements Translation: Translate ambiguous client requirements into robust agent architectures, clearly advising stakeholders when an agentic pattern is-or is not-the right solution.
• Stakeholder Communication: Interface directly with client stakeholders and articulate technical trade-offs clearly to non-technical audiences.
• Tooling Transfer & Documentation: Contribute reusable harness components to shared internal tooling and document design decisions to meet audit, validation, and handover standards.

Benefits

• Employment-based cooperation
• Comprehensive insurance for you and your family (health, life, and accident)
• Paid PTO policy (vacation, sick leaves, and public holidays)
• Tech equipment provided
• Udemy courses, workshops, trainings & expert knowledge-sharing
• Flexible hours & work setup