{"id":3409131,"url":"https://alion.io/job/autodesk-principal-data-engineer-pyspark-ai","title":"Principal Data Engineer (PySpark, AI)","company":{"id":81,"name":"Autodesk","domain":"autodesk.com","url":"https://alion.io/company/autodesk","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":98,"open_postings":128,"ghost_share":0.008,"stale_share":0.047,"repost_share":0.055,"time_to_fill_p50_days":29,"computed_at":"2026-10-30T05:48:23Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":36000,"max_usd":76000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Copilot","optional":false},{"name":"Cursor","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"AI Agents","optional":true},{"name":"Apache Iceberg","optional":true},{"name":"LLM","optional":true},{"name":"Looker","optional":true},{"name":"Power BI","optional":true}],"status":"live","first_seen_at":"2026-10-28T15:28:39Z","employer_posted_date":null,"last_verified_at":"2026-10-28T15:28:39Z","board_verified":false,"closed_at":null,"days_open":2,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":2},"description":"Job Requisition ID #\n26WD101628\n\nPosition Overview\nAutodesk is looking for a highly skilled Principal Data Engineer to join our Advanced Compliance Products (ACP) team. ACP is a strategic initiative focused on detecting non-compliant usage of Autodesk software through large-scale data processing and analytics.\nAs a Principal Data Engineer, you will drive the technical architecture, scalability, and modernization of ACP's data engineering platform, leveraging modern cloud technologies and AI-driven engineering practices. You will solve complex engineering challenges, establish best practices, and lead cross-functional technical initiatives while remaining hands-on.\n\nResponsibilities\nDefine and drive the architecture of scalable, reliable, and cost-efficient data platforms processing billions of records.\nLead the design, development, and optimization of large-scale data pipelines using Spark, PySpark, Python, and SQL.\nDrive platform modernization initiatives, including incremental processing, reusable frameworks, and modern data lakehouse architectures.\nLeverage AI/GenAI technologies and AI-assisted development tools to improve engineering productivity, automate workflows, enhance data quality, and optimize pipeline operations.\nIdentify and resolve complex performance, scalability, reliability, and data quality challenges.\nEstablish engineering best practices for CI/CD, automated testing, observability, and cloud infrastructure.\nLead technical design discussions, influence architectural decisions, and mentor engineers across teams.\nCollaborate with product managers, data scientists, and business stakeholders to translate business requirements into scalable data solutions.\nTake end-to-end technical ownership of complex, cross-team engineering initiatives.\n\nMinimum Qualifications\n10+ years of experience in data engineering, distributed systems, or large-scale data processing.\nStrong hands-on expertise in Apache Spark, PySpark, Python, Spark SQL, and SQL, including performance tuning and optimization.\nProven experience architecting and scaling complex data pipelines and distributed data processing systems.\nStrong knowledge of AWS, Hive, Snowflake, and modern data lakehouse architectures.\nExperience with data modeling, ETL/ELT frameworks, orchestration tools (such as Airflow), and CI/CD practices.\nHands-on experience with AI/GenAI technologies, including LLMs, prompt engineering, and AI-assisted development tools such as GitHub Copilot, Cursor, or equivalent.\nPractical experience applying AI to data engineering workflows, such as code generation, automated testing, data quality validation, anomaly detection, pipeline optimization, or intelligent troubleshooting.\nStrong understanding of data quality, observability, reliability, and cloud infrastructure.\nDemonstrated technical leadership, architectural decision-making, and mentoring experience.\nExcellent problem-solving, communication, and cross-functional collaboration skills.\nBachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.\n\nPreferred Qualifications\nExperience with Apache Iceberg, incremental data processing, and modern orchestration frameworks.\nExperience modernizing legacy data platforms and optimizing large-scale Spark workloads.\nFamiliarity with BI tools such as Power BI or Looker.\nExperience with AI agents or LLM-powered automation.\nExperience building reusable data engineering frameworks and driving engineering standards across teams.\n\n#LI-SJ1\n\nLearn More / Plus d'information\n\nAbout Autodesk / À propos d'Autodesk \n\nWelcome to Autodesk! Amazing things are created every day with our software - from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.\n\nWe take great pride in our culture here at Autodesk - it's at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.\n\nWhen you're an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future Join us!\n\nBienvenue chez Autodesk! Nos logiciels créent chaque jour des choses extraordinaires: des bâtiments les plus écologiques aux voitures les plus propres, en passant par les usines les plus intelligentes et les films succès. Nous aidons les innovateurs concrétiser leurs idées, transformant non seulement la façon dont les choses sont fabriquées, mais aussi ce qui peut lêtre. \n\nNous sommes très fiers de notre culture chez Autodesk elle est au cœur de tout ce que nous faisons. Notre culture guide notre façon de travailler et de nous comporter les uns envers les autres, influence nos interactions avec nos clients et nos partenaires, et définit notre image au monde.\n\n En tant qu'Autodeskien, vous pouvez accomplir un travail significatif qui contribue bâtir un monde meilleur, conçu et réalisé pour tous. Prêt façonner le monde et votre avenir Rejoignez-nous!\n\nSalary transparency / Transparence Salariale\nSalary is one part of Autodesk's competitive compensation package. Offers are based on the candidate's experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.\nBelonging\nWe take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: \n\nIn-Person Onboarding and Identity Verification\nThis role may require in-person onboarding and/or in-person ID verification.","description_format":"text","description_chars":5732,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","CAD, PLM & BIM Software","Simulation & Digital Twin Software","Construction Management Software"],"lifecycle":[{"event":"open","at":"2026-10-28T17:09:50Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":29,"reasons":["seen:1","velocity","win:early","comp:brand"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/autodesk-principal-data-engineer-pyspark-ai","json_url":"https://alion.io/job/autodesk-principal-data-engineer-pyspark-ai.json","meta":{"generated_at":"2026-10-31T02:26:13Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3414,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-01T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":81},"rest":"https://alion.io/mcp/rest/get_company?id=81"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fautodesk-principal-data-engineer-pyspark-ai"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fautodesk-principal-data-engineer-pyspark-ai"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fautodesk-principal-data-engineer-pyspark-ai"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/autodesk-principal-data-engineer-pyspark-ai\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fautodesk-principal-data-engineer-pyspark-ai"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3409131}},{"tool":"search_jobs","arguments":{"company":"autodesk.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3409131},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}