{"id":115128,"url":"https://alion.io/job/h1-staff-data-engineer-data-lake","title":"Staff Data Engineer- Data Lake","company":{"id":15112,"name":"H1","domain":"h1.com","url":"https://alion.io/company/h1","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"A","score":95,"open_postings":3,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":82,"computed_at":"2026-10-16T05:45:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"staff","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["New York, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":190000,"max":220000,"currency":"USD","period":"year","gross":null,"usd_annual":220000},"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"Docker","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Java","optional":false},{"name":"Kubernetes","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Claude Code","optional":true},{"name":"Copilot","optional":true}],"status":"live","first_seen_at":"2026-06-03T17:42:36Z","employer_posted_date":"2026-06-03","last_verified_at":"2026-10-16T23:33:41Z","board_verified":true,"closed_at":null,"days_open":135,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":135},"description":"At H1, we believe access to the best healthcare information is a basic human right. Our mission is to provide a platform that can optimally inform every doctor interaction globally. This promotes health equity and builds needed trust in healthcare systems. To accomplish this, our teams harness the power of data and AI technology to unlock groundbreaking medical insights and convert those insights into actions that result in optimal patient outcomes and accelerate an equitable and inclusive drug development lifecycle. Visit h1.co to learn more about us.\nAs part of H1’s hiring process, all candidates are required to participate in an in-person final interview. Depending on your location, this may require travel.\nData Engineering is responsible for the development and delivery of our most important asset-our data. With thousands of data sources from around the world, the team ensures that data is accurate, normalized, and delivered at a velocity that keeps up with real-world changes. As we expand our markets and the scope of data we provide to our customers, our team must scale to meet that demand.\nWHAT YOU'LL DO AT H1\nAs a Staff Data Engineer on the Data Lake team at H1, you will play a critical role in shaping the architecture, scalability, reliability, and long-term direction of our core data platform. This role is designed for a highly technical engineer who is excited to grow into an Engineering Manager track while remaining deeply hands-on technically.\nThe Data Lake is the foundation of H1’s platform, responsible for the validation, accuracy, standardization, and quality of the data powering every downstream product and team across the organization. You will help lead the evolution of this platform while supporting and mentoring a growing team of engineers.\nYou will:\n- Architect, build, and scale distributed ETL/ELT pipelines and large-scale ingestion frameworks across structured and unstructured healthcare datasets.\n- Lead the evolution of H1’s Data Lake architecture with a focus on scalability, observability, reliability, and cost optimization.\n- Own and improve data quality, validation, normalization, and standardization workflows across thousands of global data sources.\n- Design and optimize batch and near real-time data processing frameworks using cloud-native distributed systems.\n- Optimize distributed compute and storage systems, including Spark workloads, query performance, partitioning strategies, and infrastructure efficiency.\n- Drive improvements in monitoring, governance, operational excellence, and production reliability across the platform.\n- Troubleshoot complex production data and infrastructure issues across distributed systems.\n- Partner closely with Product, Infrastructure, Security, Compliance, and downstream engineering teams to support scalable and secure data delivery.\n- Mentor engineers through technical leadership, architecture reviews, and engineering best practices.\n- Help define technical roadmap priorities and contribute to long-term platform strategy and execution planning.\n- Support production operations, incident response, and platform health as part of overall ownership of the Data Lake ecosystem.\nABOUT YOU\nYou are a highly technical data engineer who thrives in lean, high-ownership environments and enjoys solving complex distributed systems challenges. You are excited by the opportunity to influence technical direction, mentor engineers, and grow into broader engineering leadership responsibilities while remaining hands-on.\n- You have deep experience designing and scaling distributed data platforms and large-scale pipelines in cloud-native environments.\n- You excel at building reliable, observable, and maintainable data systems supporting critical business and analytics workloads.\n- You have strong expertise in distributed processing, performance optimization, and modern data architecture patterns.\n- You are comfortable leading technical initiatives and influencing architecture decisions across teams.\n- You communicate effectively with both technical and non-technical stakeholders.\n- You enjoy mentoring engineers and helping raise the engineering bar across teams.\n- You are energized by ownership, autonomy, and solving ambiguous technical challenges.\nREQUIREMENTS\n- 8+ years of experience in data engineering, software engineering, or related fields with significant experience building and scaling distributed data platforms.\n- Demonstrated technical leadership experience with interest in or experience mentoring and leading engineers.\n- Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages.\nAdvanced SQL expertise, including performance tuning and optimization across large datasets.\n- Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar).\n- Experience designing and scaling modern cloud-native data lake architectures and large-scale ingestion frameworks.\n- Experience with orchestration and workflow management tools such as Argo, Airflow, or similar technologies.\n- Strong understanding of distributed storage systems, partitioning strategies, and file formats such as Parquet, Avro, and ORC.\n- Experience with Docker, Kubernetes, and modern containerization technologies.\n- Experience implementing monitoring, observability, and data quality frameworks within production environments.\n- Experience with large-scale data cleaning, parsing, normalization, and validation workflows preferred.\n- Experience working with healthcare, life sciences, publication, or large-scale entity-resolution datasets preferred.\n- Exposure to ML/AI-driven data enrichment, parsing, or validation workflows is a plus.\n- Experience using AI-assisted coding tools (e.g., GitHub Copilot, Claude Code) to accelerate development while maintaining quality is encouraged\nCOMPENSATION\nThis role pays $190,000 to $220,000 per year, based on experience, in addition to stock options.\nAnticipated role close date: 9/15/2026","description_format":"text","description_chars":6045,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Stock options"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Health Data & Interoperability","Clinical Trial Sites & Recruitment","Pharma Marketing & Medical Communications"],"lifecycle":[{"event":"open","at":"2026-06-03T17:42:36Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 5 · green card filings: 2","filings_12m":5,"filings_prev_12m":6,"green_card_filings_12m":2,"median_offered_wage_usd":130357,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":1}],"liveness":{"score":22,"band":"cold","label":"Long shot","p_open":1,"p_active":0.606,"p_room":0.36,"age_days":134,"expected_fill_days":82,"reasons":["conf:3","win:tail","crowd:"],"computed_at":"2026-10-16T05:45:00Z"},"pay":{"stated_usd_annual":220000,"is_top_pay":true},"html_url":"https://alion.io/job/h1-staff-data-engineer-data-lake","json_url":"https://alion.io/job/h1-staff-data-engineer-data-lake.json","meta":{"generated_at":"2026-10-16T23:43:48Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":5808,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":15112},"rest":"https://alion.io/mcp/rest/get_company?id=15112"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fh1-staff-data-engineer-data-lake"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fh1-staff-data-engineer-data-lake"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fh1-staff-data-engineer-data-lake"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/h1-staff-data-engineer-data-lake\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fh1-staff-data-engineer-data-lake"}]}