{"id":3870978,"url":"https://alion.io/job/blackline-principal-software-engineer","title":"Principal Software Engineer","company":{"id":1833518,"name":"BlackLine","domain":"blackline.com","url":"https://alion.io/company/blackline-com","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":100,"open_postings":5,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":48,"computed_at":"2026-11-07T05:47:08Z"}},"role":"Backend","role_family":"Backend","seniority":"lead","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":18,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Anomaly Detection","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Claude","optional":false},{"name":"Fine-tuning","optional":false},{"name":"GCP","optional":false},{"name":"Gemini","optional":false},{"name":"Google Cloud Run","optional":false},{"name":"Google GKE","optional":false},{"name":"GPT-4","optional":false},{"name":"gRPC","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"Kubernetes","optional":false},{"name":"LangGraph","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Multi-Agent Systems","optional":false},{"name":"pgvector","optional":false},{"name":"Pinecone","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"RAG","optional":false},{"name":"Rest API","optional":false},{"name":"Vertex AI Agent Builder","optional":false},{"name":"Weaviate","optional":false},{"name":"LangChain","optional":true},{"name":"PostgreSQL","optional":true}],"status":"live","first_seen_at":"2026-11-06T14:25:02Z","employer_posted_date":"2026-11-06","last_verified_at":"2026-11-07T08:35:50Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Make Your Mark:\nAs a Principal Software Engineer, you will play a crucial role in delivering high quality releases to our customers by defining application architecture, designing, developing, troubleshooting, maintaining, optimizing, and scaling BlackLine's cloud-based backend services and driving innovation into our product suite. In this role, you will work with management to identify strategic directions for system architecture. You will research, test, benchmark and evaluate new technologies, propose scaling solutions, collaborate with other teams to define, and build new features, optimize existing features while writing elegant code, coaching, and mentoring less experienced software engineers and communicating with people at all levels. In this position, every bit of your software design skill be drawn upon in order to drive towards and implement best of breed features in a rapid iterative environment. Your attention to detail, tenacity, and creative problem-solving and thinking ahead will directly impact the company's success, as well as the success of your peers across the organization. If you have a passion for delivering products that make a difference, are driven to make things better in all that you do and are eager to work in a collaborative agile environment where Developers are dedicated to solving customer problems and innovating, you will be a great fit for the team.You'll Get To:\nWe are looking for an accomplished, hands-on Principal Engineer with 18+ years of software engineering experience and a strong record of setting technical direction in Agentic AI, Generative AI systems and Software/Data engineering. This is the most senior individual contributor role on the Agentic AI platform. You will set the multi-year technical vision and own the architecture across products, while staying close enough to the code to make the critical calls. You will guide Senior Staff Engineers, Staff Engineers and Technical Leads across several teams.\nThe ideal candidate has led 5+ enterprise-scale AI programs from strategy and solution design through to production at scale, and has shaped platform architecture that several product teams build on. You can define an organization-wide AI architecture strategy and present it to executives, and you can also prototype and write production-grade Python when a problem needs it. You bring the vision to set direction across teams, the judgment to make high-stakes architecture and investment decisions, and the credibility to raise the engineering bar across the company.\nThe role spans strategy, architecture and execution. You will set the AI platform roadmap, define the governance and trust frameworks that make agents enterprise-ready on a secure, compliant financial platform, and establish the engineering standards that every AI product team follows.\nKey Responsibilities\nTechnical Strategy & Vision\nDefine and own the multi-year technical strategy and architecture roadmap for the Agentic AI platform across all product lines.\nMake build-vs-buy, technology selection and platform investment decisions, and present the trade-offs, risks and ROI to executive leadership.\nTrack the agentic AI landscape: frameworks, protocols, models, regulation. Turn those shifts into strategic bets and platform direction.\nIdentify new opportunities where AI can deliver business value across product areas, and shape them into fundable programs.\nArchitecture Leadership\nOwn the end-to-end architecture of the Agentic AI system: agent registry, policy engine, metrics ingestion pipeline, anomaly detection and HITL orchestration.\nDefine the reference architecture for multi-agent systems: trust tier boundaries (T0-T3), separation of concerns, fault isolation and full observability.\nSet organization-wide patterns for data flows, integrations (REST, gRPC, SSE, event-driven) and storage strategies for AI workloads in a multi-tenant FinTech environment.\nOwn the design principles for durable DAG execution, pause and resume of long-running financial close workflows, event-sourced audit trails and SLA-tracked HITL approval gates.\nRun the architecture governance process: ADRs, cross-team design reviews and production readiness assessments for critical systems.\nTechnical Leadership & Organizational Influence\nSplit the platform into programs and workstreams. Assign ownership to Senior Staff and Staff Engineers, with clear API contracts, interface definitions and success metrics.\nAct as the senior technical authority for Engineering, Product, Security, Compliance, Finance and Data Platform leaders, and align architecture decisions across organizations.\nRepresent the AI Platform in executive and cross-functional technical forums, and externally at industry events, through publications and in partner discussions.\nFind and resolve cross-team technical dependencies and systemic risks early in planning.\nMentorship & Talent Development\nGrow Senior Staff Engineers, Staff Engineers and Technical Leads into the next generation of technical leaders through architecture working sessions, design clinics and 1:1 coaching.\nReview designs from senior engineers. Find gaps in governance, observability or compliance thinking and guide them to better solutions rather than overriding them.\nShape technical hiring standards and help attract top senior talent to the organization.\nEngineering Standards & Governance\nDefine and drive adoption of engineering standards across all AI product teams: agent identity patterns, audit trail requirements, HITL design principles, Kafka topic design and eval harness setup.\nSet the quality bars: performance benchmarks, eval thresholds, observability standards and security posture.\nOwn the agent governance and certification framework, including the T1 agent certification pipeline and its evaluation harnesses for model quality, factuality and safety at scale.\nDrive the production rollouts and release readiness for all releases\nSupport the on-call and be the first point of contact for any incidents reported internally or from customers\nHands-on Technical Depth\nStep in directly on the hardest problems with code, prototypes, design sketches or paired debugging: novel system design, complex integrations, critical production incidents. This is not a review-only role.\nGuide the design and delivery of intelligent agents for complex, multi-step financial reasoning, using ReAct, Plan-and-Execute and other advanced reasoning strategies.\nSet the orchestration strategy across LangGraph, Temporal, Google Workflows or equivalent, with documented architectural reasoning.\nDefine deployment architecture on GCP (Vertex AI Agent Builder, Cloud Run, GKE) or AWS/Azure equivalents, including Kubernetes namespace isolation for T1 certified partner agents.\nShape MCP (Model Context Protocol) server and client patterns for external agent integration (T2/T3 trust tiers via SSE/HTTPS).\nGuide RAG architecture with vector stores (Pinecone, Weaviate, pgvector) for retrieval over domain-specific financial documents.\nLead model strategy across foundation models (Gemini, GPT-4, Claude), covering fine-tuning, prompt engineering and evaluation for accounting tasks such as close commentary and variance explanation.\nOwn the inference cost strategy: caching, batching, model routing and quantization. Keep token costs visible in Control Tower.\nWhat You'll Bring:\nRequired\n18+ years of software engineering experience, including significant time as a senior or principal-level technical leader.\nA record of setting technical strategy and architecture across multiple teams or product lines.\nLed 5+ enterprise-scale AI/ML or GenAI programs from strategy through to production at scale.\nDeep expertise in agentic AI architectures, multi-agent orchestration, LLM evaluation and AI governance.\nStrong hands-on proficiency in Python and distributed systems design.\nExtensive experience with cloud-native architecture on GCP, AWS or Azure, including Kubernetes and event-driven systems such as Kafka.\nProven ability to influence executives and align technical direction across Engineering, Security, Compliance and Business stakeholders.\nExperience mentoring Senior Staff and Staff Engineers into senior technical leaders.\nPreferred\nExperience building AI systems in regulated industries: FinTech, financial close, accounting or audit.\nFamiliarity with SOX, audit trail and compliance-by-design patterns.\nIndustry visibility through patents, publications, conference talks or open-source contributions.\nThrive at BlackLine Because You Are Joining:\nA technology-based company with a sense of adventure and a vision for the future. Every door at BlackLine is open. Just bring your brains, your problem-solving skills, and be part of a winning team at the world's most trusted name in Finance Automation!\nA culture that is kind, open, and accepting. It's a place where people can embrace what makes them unique, and the mix of cultural backgrounds and varying interests cultivates diverse thought and perspectives.\nA culture where BlackLiner's continued growth and learning is empowered. BlackLine offers a wide variety of professional development seminars and inclusive affinity groups to celebrate and support our diversity.\nBlackLine is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity or expression, race, ethnicity, age, religious creed, national origin, physical or mental disability, ancestry, color, marital status, sexual orientation, military or veteran status, status as a victim of domestic violence, sexual assault or stalking, medical condition, genetic information, or any other protected class or category recognized by applicable equal employment opportunity or other similar laws.\nBlackLine recognizes that the ways we work and the workplace itself has shifted. We innovate in a workplace that optimizes a combination of virtual and in-person interactions to maximize collaboration and nurture our culture. Candidates who live within a reasonable commute to one of our offices will work in the office at least 3 days a week.\nAccommodations:\nBlackLine is committed to creating an inclusive and accessible experience for all candidates. If you require a reasonable accommodation that would better enable your success during the application or interview process, please complete this form.","description_format":"text","description_chars":10389,"description_truncated":false,"requirements":{"experience_years_min":18,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Professional development"],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Accounting & Tax Software"],"lifecycle":[{"event":"open","at":"2026-11-06T14:25:02Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":0,"expected_fill_days":48,"reasons":["conf:10","velocity","win:early"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/blackline-principal-software-engineer","json_url":"https://alion.io/job/blackline-principal-software-engineer.json","meta":{"generated_at":"2026-11-07T12:07:34Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13179,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1833518},"rest":"https://alion.io/mcp/rest/get_company?id=1833518"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fblackline-principal-software-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fblackline-principal-software-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fblackline-principal-software-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/blackline-principal-software-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fblackline-principal-software-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3870978}},{"tool":"search_jobs","arguments":{"company":"blackline.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3870978},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}