{"id":2642889,"url":"https://alion.io/job/datapelago-principal-data-processing-engineer","title":"Principal Data Processing Engineer","company":{"id":4422787,"name":"DataPelago","domain":"datapelago.ai","url":"https://alion.io/company/datapelago","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"BambooHR","truth_index":{"grade":"D","score":40,"open_postings":12,"ghost_share":1,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-17T05:46:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Mountain View, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":148000,"max_usd":291000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":390},"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"Apache Doris","optional":false},{"name":"C++","optional":false},{"name":"CI/CD","optional":false},{"name":"Flink","optional":false},{"name":"Linux","optional":false},{"name":"Machine Learning","optional":false},{"name":"Ray","optional":false},{"name":"Rust","optional":false},{"name":"Spark","optional":false}],"status":"live","first_seen_at":"2025-10-28T00:00:00Z","employer_posted_date":"2025-10-28","last_verified_at":"2026-10-17T23:22:22Z","board_verified":true,"closed_at":null,"days_open":354,"trust":{"level":"ghost","repost_count":0,"flags":["stale","company_stale"],"days_open":354},"description":"Principal Data Processing Engineer\nMountain View, CA\nAbout DataPelago:\nDataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.\nThe Role:\nAs a Principal Data Processing Engineer, you will be a key technical leader in the development of the core execution components of our data processing engine. You will lead architecture, design, and implementation that will enhance the functional breadth, performance, scale, and reliability of the engine to deliver a product that will redefine how users extract intelligence from their data. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.\nWhat You'll Do:\nDrive the evolution of our parallel and distributed execution engine architecture, with a strong focus on leveraging accelerated computing technologies.\nEnd-to-End Ownership: Lead the execution engine team in the complete lifecycle of design,implementation, and rollout of an enterprise-grade product.\n\nIndividually design, implement, test, and maintain critical components of the data processing execution engine.\nInnovation and Differentiation: Analyze technology advances from industry and academia to identify opportunities for the engine to enhance technology and product leadership.\nCollaboration: Partner effectively with engineering, product management,and customer success teams. \n\nGuide and mentor engineers on the execution engine team.\nFoster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain the highest product quality, security, efficiency, and productivity.\nWhat You'll Bring:\nBS/MS in Computer Science (or a related field) with 10+ years of relevant experience \n7+ years of deep technical experience in developing core components of enterprise-grade database or analytics execution engines designed for large-scale data processing.\nProven expertise in developing high-performance parallel implementations of data processing operators and functions on rich data types.\nSignificant experience developing for platforms such as Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion/Comet preferred.\nPrevious experience working as technical lead/architect with teams of 10+ engineers in the design, development, and successful release of high-performance data processing engines for large production deployments.\nProficiency in C, C++, and Rust programming.\nExtensive development experience in Linux environments.\nStrong analytical and problem-solving skills with a passion for performance optimization.\nExcellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.\nWhy Join DataPelago?\nTechnical Leadership: Take a leadership role in shaping the architecture and development of how our core engine works with open source data processing platforms\nCutting-Edge Innovation: Work on challenging problems at the forefront of acceleratedcomputing and data processing.\n\nSignificant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.\nMentorship and Growth: Mentor and guide other talented engineers while expanding your own technical expertise.\nCompetitive compensation, stock options, comprehensive benefits package, and leadership developmentopportunities","description_format":"text","description_chars":3934,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Stock options"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-15T19:23:25Z"}],"visa":[],"liveness":{"score":4,"band":"cold","label":"Long shot","p_open":1,"p_active":0.153,"p_room":0.28,"age_days":354,"expected_fill_days":26,"reasons":["conf:3","stale_co","ghost","win:tail","crowd:"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/datapelago-principal-data-processing-engineer","json_url":"https://alion.io/job/datapelago-principal-data-processing-engineer.json","meta":{"generated_at":"2026-10-17T23:31:11Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1829,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4422787},"rest":"https://alion.io/mcp/rest/get_company?id=4422787"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fdatapelago-principal-data-processing-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fdatapelago-principal-data-processing-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fdatapelago-principal-data-processing-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/datapelago-principal-data-processing-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fdatapelago-principal-data-processing-engineer"}]}