{"id":2788192,"url":"https://alion.io/job/tech-mahindra-sr-data-engineer-python-pyspark-cloud","title":"Sr. Data Engineer - Python + Pyspark + Cloud","company":{"id":5484,"name":"Tech Mahindra","domain":"techmahindra.com","url":"https://alion.io/company/tech-mahindra","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Career site","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":22000,"max_usd":44000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":59},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Flink","optional":false},{"name":"Java","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"AI Agents","optional":true},{"name":"CI/CD","optional":true},{"name":"RAG","optional":true}],"status":"live","first_seen_at":"2026-10-17T17:06:49Z","employer_posted_date":"2026-10-17","last_verified_at":"2026-10-17T23:44:37Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Job Description\n\nSkill Set :\nSQL,cloud,pyspark\n\nTotal Experience :\n5.00 to 10.00 Years\n\nNo of Openings :\n1\n\nJob Post Date :\n02/09/2026\n\nJob Expiry Date :\n31/10/2026\n\nDomain :\nIT\n\nLocation :\nHYDERABAD [India]\n\nJob Reference No :\n4105167\n\nJob Summary\nKey Responsibilities\n•Design and deliver Oracle AI Data Platform solutions, including platform implementations and migrations.\n•Build scalable, governed, AI ready data pipelines, software components, and services for data consolidation, integration, transformation, and data products.\n•Develop Lakehouse, Medallion, streaming, and real time analytics solutions on OCI using Spark, PySpark, Delta Lake, Parquet, Flink, and distributed systems patterns.\n•Build AI/ML and Generative AI data pipelines, including feature engineering, feature extraction, MLOps workflows, and vector ready datasets.\n•Assess and modernize workloads by refactoring PySpark and ETL code and optimizing Spark configuration, partitioning, memory, adaptive query execution, reliability, and performance.\n•Develop APIs, SDKs, web applications, and automation frameworks that simplify adoption of Oracle AI Data Platform and Generative AI services.\n•Contribute to reusable migration accelerators, reference architectures, automation frameworks, and delivery methodologies.\n•Contribute to engineering excellence across CSS and continuously evaluate emerging AI, cloud, and data engineering practices.\nMinimum Qualifications\n•5‿8 years of experience designing, building, and delivering scalable data platforms.\n•Proven experience as a Data Engineer, Software Engineer, AI Engineer, Consulting Engineer, or similar technical role.\n•Strong knowledge of modern cloud data platforms like Databricks.\n•Strong hands on experience with Apache Spark, PySpark, Delta Lake, Python, SQL, and modern data ecosystems; experience with Scala, Java, or Kafka is a plus.\n•Exposure to building AI agents and orchestration pipelines.\n•Strong understanding of Spark architecture, configuration, distributed systems, performance optimization, and tuning.\n•Experience modernizing analytics platforms and refactoring Spark and ETL workloads.\n•Working understanding of networking concepts to support data connectivity, configuration, and troubleshooting.\n•Working knowledge of DataOps, CI/CD, Infrastructure as Code, data security, governance, metadata, lineage, and observability.\n•Strong communication, customer collaboration, and stakeholder management skills.\nPreferred Qualifications\n•OCI Architect, Data Engineering, Data Science, or AI certifications.\n•Experience with LLMs, Retrieval Augmented Generation (RAG), vector databases, AI agents, and MLOps.\n•Experience contributing to large scale data platform modernization programs and distributed engineering teams.\n•Industry experience in Financial Services, Healthcare, Public Sector, Telecommunications, Manufacturing, or Retail.\nDelivery & Collaboration Expectations\n•Contribute to the technical delivery strategy for customer programs.\n•Share field feedback with Oracle product and engineering teams.\n•Apply and improve reusable engineering standards and delivery frameworks.\n•Build expertise in Oracle cloud data engineering and AI platforms.\n•Collaborate with senior architects and engineers across Oracle CSS.\n•Participate in architecture reviews and customer innovation workshops.","description_format":"text","description_chars":3396,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","IT Outsourcing & Dedicated Teams","Systems Integrators","Business Process Outsourcing (BPO)"],"lifecycle":[{"event":"open","at":"2026-10-17T17:06:49Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":26,"reasons":["conf:0","win:early","comp:brand"],"computed_at":"2026-10-17T23:53:38Z"},"pay":null,"html_url":"https://alion.io/job/tech-mahindra-sr-data-engineer-python-pyspark-cloud","json_url":"https://alion.io/job/tech-mahindra-sr-data-engineer-python-pyspark-cloud.json","meta":{"generated_at":"2026-10-17T23:53:38Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2318,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":5484},"rest":"https://alion.io/mcp/rest/get_company?id=5484"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftech-mahindra-sr-data-engineer-python-pyspark-cloud"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftech-mahindra-sr-data-engineer-python-pyspark-cloud"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftech-mahindra-sr-data-engineer-python-pyspark-cloud"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/tech-mahindra-sr-data-engineer-python-pyspark-cloud\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftech-mahindra-sr-data-engineer-python-pyspark-cloud"}]}