{"id":337380,"url":"https://alion.io/job/karya-research-intern-ai-evaluations","title":"Research Intern - AI Evaluations","company":{"id":63591,"name":"Karya","domain":"karya.in","url":"https://alion.io/company/karya","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"C","score":67,"open_postings":18,"ghost_share":0.556,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-26T05:57:25Z"}},"role":"Early Careers","role_family":"Early Careers","seniority":"intern","employment_type":"internship","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":8500,"max_usd":16000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":829},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Human-in-the-Loop","optional":false}],"status":"live","first_seen_at":"2026-09-04T06:17:58Z","employer_posted_date":"2026-09-04","last_verified_at":"2026-10-27T03:08:23Z","board_verified":true,"closed_at":null,"days_open":52,"trust":{"level":"ok","repost_count":0,"flags":["company_stale"],"days_open":51},"description":"About Karya:\nWhy was Karya on the cover of the Time Magazine , highlighted by Satya Nadella , and chosen by Google as its partner for Project Vaani?\nIn part, because Karya is on a mission to provide AI enabled earning and learning opportunities to economically underserved communities, thereby building a pathway out of poverty for them. Karya achieves this while also delivering high quality, timely, and price competitive data to its clients.\nKarya's workers make at least 20 times the Indian minimum wage and through our one-of-a-kind digital work platform, we have delivered over 40 million digital tasks and are poised to positively impact over 100 thousand workers by the end of the year. In the coming years, our goal is to rapidly scale our impact by bringing economic opportunities to millions of underserved users in India.\nThe Role\nAt Karya, we're building AI evaluation systems that measure and understand whether these systems actually work for the communities they're meant to serve. We're looking for a research intern who can bring a social science lens into this work, specifically around evaluating language technologies (speech, text, and translation systems) built for and with Indian communities.\nYou'll be embedded with the Evaluations team within Karya's tech team, which means you'll get a front-row seat to the full arc of AI work happening in India today from how models are built, to how they're tested, to how they land in the hands of rural and economically underprivileged communities. You'll be expected to stay current on AI evaluation science itself, and use your disciplinary training to ask sharper questions about what these systems are actually doing, and to whom.\nConcretely, this means:\nApplying social science research methods (qualitative and/or quantitative) to the evaluation of AI/ML systems, particularly language technologies\nContribute to developing AI evaluation and testing methodologies\nInvestigating how these technologies interact with and affect the communities they're deployed among, especially in rural and low-resource contexts in India\nWorking closely with the Evaluations team to understand both the \"build\" and \"evaluate\" sides of AI systems\nThis role is well suited to someone who's curious about tech and wants to put their social science training into practice. You'll be working alongside engineers, bringing a rigorous footing and a social science researcher's lens into a technical space. If you're a social science student or researcher who's been eyeing the AI/tech space and wondering how to get in, this is that entry point. You'll work on live evaluation pipelines, with communities across India, on low- and high-resource language technologies, alongside a team that treats social science expertise as core to the work. \nQualifications\nEducation\nCurrently pursuing or recently completed an undergraduate or master's degree in a social science discipline (Sociology, Anthropology, Development Studies, Gender Studies, Political Science, Economics, or Science and Technology Studies (STS)\nNo technical degree required, but demonstrated interest in and conceptual understanding of AI/ML systems is expected. E.g., through self-directed learning, coursework, or prior projects at the intersection of technology and society\nSkills and Competencies\nStrong grounding in social science research methods qualitative, quantitative, or mixed (not a hard requirement, but research training expected)\nComfort conducting or supporting focus group discussions (FGDs) and field visits\nConceptual understanding of AI/ML systems - applied AI/ML literacy - enough to engage meaningfully with concepts like model evaluation, human-in-the-loop systems, and language technologies (you don't need to code, but you should be comfortable with how these systems are built and tested)\nFamiliarity with socio-technical systems thinking (i.e., understanding technology and society as mutually shaping, not separate concerns)\nMultilingual proficiency in Indian languages is a strong plus\nBackground in computational social science is a plus, not a requirement\nStrong written and verbal communication skills\nComfort working in ambiguous, early-stage research settings\nPeople matter at Karya and these are some of the perks and benefits we created for our team:\nFlexible vacation and leave policy\nFlexible work options\nInsurance as per industry standards\nAccess to industry stalwarts and networking opportunities\nKarya is an equal opportunity and affirmative action employer.\nQualified applicants will receive consideration without regard to their race, colour, religion, sex, sexual orientation, gender identity and disability.\nKarya invites all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search to apply for a career opportunity, reach out at .","description_format":"text","description_chars":4906,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":false},"security_clearance":false,"languages":[]},"benefits":["Flexible schedule"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","AI Training Data & Annotation"],"lifecycle":[{"event":"open","at":"2026-09-05T17:50:37Z"}],"visa":[],"liveness":{"score":8,"band":"cold","label":"Long shot","p_open":1,"p_active":0.299,"p_room":0.28,"age_days":51,"expected_fill_days":24,"reasons":["conf:2","stale_co","velocity","win:tail","crowd:junior"],"computed_at":"2026-10-26T05:57:25Z"},"pay":null,"html_url":"https://alion.io/job/karya-research-intern-ai-evaluations","json_url":"https://alion.io/job/karya-research-intern-ai-evaluations.json","meta":{"generated_at":"2026-10-27T04:41:57Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4273,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-28T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":63591},"rest":"https://alion.io/mcp/rest/get_company?id=63591"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fkarya-research-intern-ai-evaluations"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fkarya-research-intern-ai-evaluations"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fkarya-research-intern-ai-evaluations"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/karya-research-intern-ai-evaluations\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fkarya-research-intern-ai-evaluations"}]}