{"id":1987094,"url":"https://alion.io/job/netapp-principal-software-engineer-data-processing-optimization","title":"Principal Software Engineer, Data Processing Optimization","company":{"id":58700,"name":"NetApp","domain":"netapp.com","url":"https://alion.io/company/netapp","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Eightfold","truth_index":{"grade":"A","score":90,"open_postings":25,"ghost_share":0,"stale_share":0.4,"repost_share":0,"time_to_fill_p50_days":54,"computed_at":"2026-10-13T05:45:00Z"}},"role":"Backend","role_family":"Backend","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Jose, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":227800,"max":338800,"currency":"USD","period":"year","gross":null,"usd_annual":338800},"salary_estimate":null,"experience_years_min":15,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"Flink","optional":false},{"name":"Machine Learning","optional":false},{"name":"Presto","optional":false},{"name":"Python","optional":false},{"name":"Ray","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Trino","optional":false}],"status":"live","first_seen_at":"2026-09-19T15:09:51Z","employer_posted_date":"2026-10-06","last_verified_at":"2026-10-14T02:12:34Z","board_verified":true,"closed_at":null,"days_open":24,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":24},"description":"Job Summary\nDataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.\nDataPelago Nucleus is a universal data processing engine that is designed to accelerate the processing of diverse data - structured through unstructured - with any parallel processing framework - e.g., Apache Spark - on any infrastructure - including vectorized CPU and GPU. DataPelago Accelerator for Spark (DPA-S), our product based on DataPelago Nucleus, is running large-scale production applications of many globally renowned customers.\nThe Opportunity:\nAs a Senior/Principal Data Processing Optimization Engineer, you will be a key individual contributor in advancing planner and optimization capabilities so that DataPelago acceleration delivers the most performance and cost reduction for open-source software SQL and Python processing engines running on CPU-based and GPU-based infrastructure. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.\nWhat You'll Do:\nResearch, design, implement, and deliver logical and physical query plan optimization techniques for massively parallel data processing engines. Develop query planning and query optimization techniques that enable industry-leading performance and performance efficiency on execution engines based on heterogeneous accelerated compute elements.\nIntegrate DataPelago data processing acceleration and plan optimization capabilities with diverse open-source engines including Apache Spark and Apache Flink.\nEnsure product performance on customers’ real deployment environment and workload.\nEstablish metrics for plan quality, workload performance, reliability and lead the team in ensuring these metrics are continuously improved and achieved in customer deployments.\nProvide technical leadership through all phases of product development and post-production support. Own development and support of major components and capabilities.\nCollaborate with other engineering team members and product management in defining product features, architecture, interfaces, and development timelines. Mentor junior team members.\nWhat You'll Bring\nBachelor’s degree in computer science. Master’s or PhD preferred, especially in areas related to database systems.\n15+ years of experience developing query planning and query optimization techniques for enterprise data processing software platform. Prefer at least 5+ years of that experience on parallel platforms such as Apache Spark, Presto/Trino, or Apache Flink.\nDeep knowledge of SQL semantics and expertise in SQL processing. Additional Python processing preferred. Solid understanding of databases, data warehouses, query engines.\nDeep knowledge of query optimization state of the art, from published literature and open-source software. Prefer own open-source contributions and/or publications in this area.\nSolid understanding of the architecture, deployment, and operations of data processing platforms in applications such as data engineering, data preparation, and analytics.\nDemonstrated proficiency in all phases of software development and post-production support of enterprise software or SaaS. Adept in adopting AI effectively.\nExperience owning the definition and delivery of major product capabilities through technical leadership and collaboration spanning multiple teams.\nStrong collaboration and communication skills.\nCompensation:\nThe target salary range for this position is 227,800 - 338,800 USD. The salary offered will be determined by the candidate's location, qualifications, experience, and education and may be outside of this range. Final compensation packages are competitive and in line with industry standards, reflecting a variety of factors, and include a comprehensive benefits package. This may cover Health Insurance, Life Insurance, Retirement or Pension Plans, Paid Time Off, various Leave options, Performance-Based Incentives, employee stock purchase plan, and/or restricted stocks (RSU’s), with all offerings subject to regional variations and governed by local laws, regulations, and company policies. Benefits may vary by country and region, and further details will be provided as part of the recruitment process.","description_format":"text","description_chars":4779,"description_truncated":false,"requirements":{"experience_years_min":15,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":true},"security_clearance":false,"languages":[]},"benefits":["Health insurance","Life insurance","Retirement plans"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Cybersecurity","Natural Resources","Cloud Platforms (IaaS & PaaS)","Digital Storage"],"lifecycle":[{"event":"open","at":"2026-10-07T03:11:58Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 164 · green card filings: 52","filings_12m":164,"filings_prev_12m":207,"green_card_filings_12m":52,"median_offered_wage_usd":157315,"route":null,"cap_exempt":false,"checked_at":"2026-10-03T21:08:04+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":132}],"liveness":{"score":80,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.885,"p_room":0.9,"age_days":23,"expected_fill_days":54,"reasons":["conf:2","velocity","win:mid","comp:brand"],"computed_at":"2026-10-13T05:45:00Z"},"pay":{"stated_usd_annual":338800,"is_top_pay":true},"html_url":"https://alion.io/job/netapp-principal-software-engineer-data-processing-optimization","json_url":"https://alion.io/job/netapp-principal-software-engineer-data-processing-optimization.json","meta":{"generated_at":"2026-10-14T03:43:24Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":5664,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":58700},"rest":"https://alion.io/mcp/rest/get_company?id=58700"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnetapp-principal-software-engineer-data-processing-optimization"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnetapp-principal-software-engineer-data-processing-optimization"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnetapp-principal-software-engineer-data-processing-optimization"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/netapp-principal-software-engineer-data-processing-optimization\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnetapp-principal-software-engineer-data-processing-optimization"}]}