{"id":2437894,"url":"https://alion.io/job/mantriai-data-engineer-pyspark","title":"Data Engineer (Pyspark)","company":{"id":3801213,"name":"MantriAI","domain":"mantriai.co","url":"https://alion.io/company/mantriai","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":14500,"max_usd":38000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":21},"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"SQL","optional":false},{"name":"Spark","optional":true}],"status":"live","first_seen_at":"2026-10-13T10:17:56Z","employer_posted_date":null,"last_verified_at":"2026-10-13T10:17:56Z","board_verified":false,"closed_at":null,"days_open":22,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":22},"description":"Job Description: Data Engineer – PySpark\n\nCompany: ZAGENO\nRole: Data Engineer – PySpark / Data Platform\nExperience: 3+ years\nEmployment Type: Full-time\nPrimary Skills: Python, SQL, PySpark, Databricks, Delta Lake, Apache Airflow, Change Data Capture (CDC)\nLocation: Bangalore\nNotice Period Preference: Candidates who can join within 30 days are preferred.\n\nAbout ZAGENO\nZAGENO is a digital platform focused on simplifying the procurement of life-sciences research products. Reliable, accurate, and well-structured product data is essential to helping researchers and life-sciences organizations discover and procure the products they need. \nWe are looking for a hands-on Data Engineer with strong PySpark expertise and proven production pipeline ownership to strengthen the reliability, scalability, and correctness of our data platform. \nIn this role, you will work on the data pipelines and systems that power our life-sciences product catalog. You will address existing pipeline failures and technical debt, build efficient data synchronisation mechanisms, improve product data quality, and develop maintainable data products for business and technical stakeholders. \nThis is an engineering role for someone who enjoys solving real production problems, taking ownership from design through deployment, and building reliable systems that scale. \nRole Overview\nAs a Data Engineer, you will own the design, development, testing, deployment, and operational reliability of production-grade data pipelines. \nYou will work with large and complex datasets using PySpark, Python, SQL, Databricks, Delta Lake, and Apache Airflow. You will also build and maintain change data capture (CDC) workflows, low-latency APIs, and data synchronisation processes. \nA key part of the role involves transforming inconsistent product and supplier data into standardised, accurate, and reusable datasets. You will collaborate with Catalog Operations, Analytics, Data Science, and other business stakeholders to translate evolving requirements into robust engineering solutions. \nWe are looking for an engineer who can independently investigate failures, identify causes, implement lasting fixes, and continuously improve data quality and system performance. \nKey Responsibilities\n\n1. Production Data Pipeline Engineering\n\nDesign, develop, maintain, and optimize scalable data pipelines using PySpark, Python, and SQL.\n\nTake primary ownership of production pipelines, from data ingestion and transformation through validation, deployment, and ongoing support.\n\nTroubleshoot unreliable pipelines, resolve recurring failures, and eliminate fragile implementations and manual workarounds.\n\nImprove pipeline performance, execution efficiency, reliability, and maintainability.\n\nImplement incremental processing, efficient transformations, and appropriate data partitioning strategies.\n\nDevelop reusable components and engineering patterns that reduce duplication and technical debt.\n\n2. Databricks and Delta Lake\n\nBuild and maintain data processing workflows on Data bricks using PySpark.\n\nWork with Delta Lake tables and implement reliable data storage, transformation, and update patterns.\n\nApply appropriate incremental processing and table optimization techniques.\n\nMaintain data consistency and correctness across pipeline runs","description_format":"text","description_chars":3307,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Biotechnology","AI Agents","Career Services"],"lifecycle":[{"event":"open","at":"2026-10-13T11:09:00Z"}],"visa":[],"liveness":{"score":53,"band":"ok","label":"Likely open","p_open":0.85,"p_active":0.838,"p_room":0.75,"age_days":21,"expected_fill_days":29,"reasons":["seen:21","velocity","win:late"],"computed_at":"2026-11-04T06:34:25Z"},"pay":null,"html_url":"https://alion.io/job/mantriai-data-engineer-pyspark","json_url":"https://alion.io/job/mantriai-data-engineer-pyspark.json","meta":{"generated_at":"2026-11-05T01:47:32Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2388,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-06T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3801213},"rest":"https://alion.io/mcp/rest/get_company?id=3801213"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmantriai-data-engineer-pyspark"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmantriai-data-engineer-pyspark"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmantriai-data-engineer-pyspark"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/mantriai-data-engineer-pyspark\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmantriai-data-engineer-pyspark"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2437894}},{"tool":"search_jobs","arguments":{"company":"mantriai.co","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2437894},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}