{"id":3333611,"url":"https://alion.io/job/citi-specialized-analytics-senior-analyst-3","title":"Specialized Analytics Senior Analyst","company":{"id":7841,"name":"Citi","domain":"citi.com","url":"https://alion.io/company/citi","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":100,"open_postings":512,"ghost_share":0.006,"stale_share":0.131,"repost_share":0.072,"time_to_fill_p50_days":9,"computed_at":"2026-11-01T06:10:25Z"}},"role":"Analytics","role_family":"Analytics","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":15500,"max_usd":31000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":25},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon S3","optional":false},{"name":"Chain-of-Thought","optional":false},{"name":"CI/CD","optional":false},{"name":"DSPy","optional":false},{"name":"FastAPI","optional":false},{"name":"Function Calling","optional":false},{"name":"Hallucination","optional":false},{"name":"Hugging Face","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"LangChain","optional":false},{"name":"LlamaIndex","optional":false},{"name":"LLM","optional":false},{"name":"NLP","optional":false},{"name":"NumPy","optional":false},{"name":"Pandas","optional":false},{"name":"Prompt Caching","optional":false},{"name":"Pydantic","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"Synthetic Data","optional":false},{"name":"Multi-Agent Systems","optional":true}],"status":"live","first_seen_at":"2026-10-27T12:49:48Z","employer_posted_date":"2026-10-27","last_verified_at":"2026-11-01T23:41:08Z","board_verified":true,"closed_at":null,"days_open":5,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":5},"description":"Job Description\nOverview\nThe C12 is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry standards and practices. Good understanding of how the team and area integrate with others in accomplishing the objectives of the sub function/ job family. Applies analytical thinking and knowledge of data analysis tools and methodologies. Requires attention to detail when making judgments and recommendations based on the analysis of information. Typically deals with variable issues with potentially broader business impact. Applies professional judgment when interpreting data and results. Breaks down information in a systematic and communicable manner. Developed communication and diplomacy skills are required to exchange potentially complex/sensitive information. Moderate but direct impact through close contact with the businesses' core activities. Quality and timeliness of service provided will affect the effectiveness of own team and other closely related teams.\nTTS Analytics C12\nWhat do we do?\nThe TTS Analytics team provides analytical insights to the Product, Pricing, Client Experience and Sales functions within the global Treasury & Trade Services business. The team works on business problems focused on enhancing client experience, driving acquisitions, cross-sell and revenue growth.\nThe team extracts relevant insights, identifies business opportunities, converts business problems into analytical frameworks, uses big data tools and AI/ML techniques to drive data driven business outcomes in collaboration with business and product partners.\nThe team works on building and operating Generative AI and deep learning solutions - Design, implement, and scale production-grade AI applications end-to-end-from data ingestion and model services to user-facing interfaces, observability, and secure deployments.\nROLE DESCRIPTION\nThe role will be AVP (C12) in the TTS Analytics team.\nThe role will involve working on\nAs a key contributor to ideation on analytical projects to tackle strategic business priorities.\nThe role will require endless curiosity, as ambiguity and open-ended questions are a core part of the team’s work.\nOwn end-to-end delivery of AI-powered products: requirements, design, implementation, testing, deployment, and support.\nDesign, build, and maintain automated, scalable evaluation pipelines and frameworks (e.g., using LLM-Eval-Harness, or custom internal frameworks).\nCurate, version, and synthetically augment high-quality ground-truth (\"golden\") datasets covering diverse use cases, edge cases, domain-specific tasks, and adversarial vectors.\nImplement metric suites evaluating task accuracy, semantic similarity, faithfulness, hallucination rates, toxicity, relevancy, bias, deterministic constraint satisfaction, and tool/agent execution fidelity.\nEstablish statistically sound LLM-as-a-Judge evaluation protocols, human-in-the-loop (HITL) calibration workflows, and deterministic/statistical baseline checks (BLEU, ROUGE, Exact Match, BERTScore).\nFormulate automated test suites and migration runbooks to transition production workloads across model versions with minimal disruption and validated parity.\nDevelop structured, reproducible prompt architectures using advanced techniques (Few-Shot, Chain-of-Thought, ReAct, DSPy, Automated Prompt Optimization).\nAnalyze and optimize token budgets, prompt caching, model routing strategies (e.g., small SLMs for routing/extraction vs. large frontier models for complex reasoning), and speculative decoding strategies.\nEnforce schema compliance (JSON schema, Pydantic, function calling / tool definitions) and deterministic constraints across varying foundation model backends.\nPartner closely with platform architects, MLOps, backend engineers, and data engineers to integrate evaluation artifacts directly into CI/CD pipelines and deployment workflows.\nLead the AI governance process\nAuthor comprehensive Object Development Documentation (ODD), Model Validation packages, model risk assessments, and technical artifacts detailing model design, limitation boundaries, training/evaluation data lineage, and testing methodologies.\nEngage early with new AI initiative owners to scope evaluation criteria, acceptable error thresholds, bias assessments, and testing requirements prior to production onboarding.\nAppropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.\nAbility to build partnerships with cross-function leaders.\nQUALIFICATIONS\nExperience:\nOverall 8+ years of experience. 5+ years of Data Science/ML engineering experience, with at least 1-2 years dedicated to Generative AI, LLMs, NLP, and evaluation frameworks.\nMust have substantial experience in:\n· Strong proficiency in Python (async/concurrency, FastAPI/Pydantic, pytest) and standard data analysis stacks (Pandas, NumPy).\n· Hands-on experience with modern LLM tooling such as LangChain, LlamaIndex, Hugging Face, etc..\n· Deep understanding of quantitative evaluation metrics (Semantic similarity, Faithfulness, Answer Relevancy, Precision/Recall, Hallucination scoring, Toxicity).\n· Experience building structured dataset curation pipelines, data cleansing, versioning (like S3), and synthetic data generation techniques.\n· Observability: application monitoring, centralized logging, tracing; strong debugging and problem-resolution skills.\n· AI Governance\nGood to Have\n· Prior experience in financial services, healthcare, or heavily regulated enterprise environments, particularly with Model Risk Governance\n· Experience evaluating complex multi-step reasoning, tool-calling agents, memory systems, and Multi-Agent Orchestration frameworks.\n· Knowledge of data privacy and responsible AI practices; experience implementing audit and guardrail tooling.\nEducation:\nBachelor / Masters (preferred) in Computer Science Engineering, Data Science, AI/ML, or related technical disciplines (or equivalent practical experience).\nThis job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.\n------------------------------------------------------\nJob Family Group:\nDecision Management------------------------------------------------------\nJob Family:\nSpecialized Analytics (Data Science/Computational Statistics)------------------------------------------------------\nTime Type:\nFull time------------------------------------------------------\nMost Relevant Skills\nPlease see the requirements listed above.------------------------------------------------------\nOther Relevant Skills\nFor complementary skills, please see above and/or contact the recruiter.------------------------------------------------------\nCiti is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.\nIf you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.\nView Citi’s EEO Policy Statement and the Know Your Rights poster.","description_format":"text","description_chars":7655,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Cards & Card Issuing","Wealth Management & Financial Advisors","Payment Processing & Gateways","Investment Banking & M&A Advisory"],"lifecycle":[{"event":"open","at":"2026-10-27T12:49:48Z"}],"visa":[],"liveness":{"score":81,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.897,"p_room":0.9,"age_days":4,"expected_fill_days":9,"reasons":["conf:5","velocity","win:mid","comp:brand"],"computed_at":"2026-11-01T06:10:25Z"},"pay":null,"html_url":"https://alion.io/job/citi-specialized-analytics-senior-analyst-3","json_url":"https://alion.io/job/citi-specialized-analytics-senior-analyst-3.json","meta":{"generated_at":"2026-11-02T00:08:06Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":233,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":7841},"rest":"https://alion.io/mcp/rest/get_company?id=7841"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fciti-specialized-analytics-senior-analyst-3"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fciti-specialized-analytics-senior-analyst-3"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fciti-specialized-analytics-senior-analyst-3"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/citi-specialized-analytics-senior-analyst-3\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fciti-specialized-analytics-senior-analyst-3"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3333611}},{"tool":"search_jobs","arguments":{"company":"citi.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3333611},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}