{"id":864070,"url":"https://alion.io/job/healthedge-senior-data-engineer-2","title":"Senior Data Engineer","company":{"id":58605,"name":"HealthEdge","domain":"healthedge.com","url":"https://alion.io/company/healthedge","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"iCIMS","truth_index":{"grade":"A","score":100,"open_postings":8,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":20,"computed_at":"2026-11-11T05:49:22Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":18500,"max_usd":40000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":75},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon EventBridge","optional":false},{"name":"Amazon S3","optional":false},{"name":"Apache Hudi","optional":false},{"name":"Apache Iceberg","optional":false},{"name":"AWS","optional":false},{"name":"AWS Glue","optional":false},{"name":"AWS Step Functions","optional":false},{"name":"Azure DevOps","optional":false},{"name":"CI/CD","optional":false},{"name":"Delta Lake","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"IAM","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Rest API","optional":false},{"name":"SQL","optional":false},{"name":"Tableau","optional":false},{"name":"AWS CDK","optional":true},{"name":"Azure","optional":true},{"name":"CloudFormation","optional":true},{"name":"GitHub Actions","optional":true},{"name":"Spark","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-09-13T12:10:00Z","employer_posted_date":"2026-11-06","last_verified_at":"2026-11-11T13:37:58Z","board_verified":true,"closed_at":null,"days_open":59,"trust":{"level":"ok","repost_count":1,"flags":[],"days_open":58},"description":"Overview\nSr Data Engineer\nPosition Summary\nWe are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery-from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight. You will work closely with onshore stakeholders to translate business requirements into reliable, well-documented data products that drive operational and strategic decision-making.\nThe ideal candidate is comfortable operating independently across the data stack, takes ownership of their work, and communicates proactively with team members across time zones.\nCore Responsibilities\nETL / Data Pipeline Development\nDesign, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA, Azure DevOps, and Zendesk into a layered data lake architecture (raw → core → presentation). \nDevelop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management. \nImplement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts. \nWork with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup. \nIntegrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures. \nTroubleshoot production pipeline failures, diagnose performance bottlenecks (OOM errors, API timeouts, DPU tuning), and implement fixes with minimal guidance. \nWrite and maintain clear technical documentation for all pipeline components. \nDashboard & Reporting Development\nBuild, publish, and maintain Amazon QuickSight dashboards and analyses that serve operational and executive audiences. \nCreate well-modeled QuickSight datasets backed by Athena queries over Iceberg tables, applying appropriate joins, calculated fields, and filters. \nTranslate business requirements and existing Tableau/Excel reports into QuickSight visuals, ensuring accuracy and usability. \nOptimize dashboard performance, evaluating SPICE vs. Direct Query modes and managing refresh schedules. \nCollaborate with stakeholders to iterate on dashboard designs, incorporating feedback and refining KPIs. \nGeneral Team Responsibilities\nParticipate in code reviews, sprint planning, and knowledge-sharing sessions with the broader BI and engineering teams. \nFollow established Git branching strategies and CI/CD practices for infrastructure and code deployment. \nMonitor data freshness and pipeline health; respond to alerts and escalate issues as appropriate. \nProactively identify opportunities to improve data quality, reduce pipeline run times, and enhance reporting capabilities. \nRequired Qualifications\n5-10+ years of professional experience in data engineering, BI development, or analytics engineering. \nStrong hands-on experience with AWS Glue (PySpark), Step Functions, S3, Athena, and IAM. \nProficiency in Python and SQL; experience writing PySpark transformations and Athena queries. \nExperience building and maintaining ETL pipelines that ingest data from REST APIs or SaaS platforms (e.g., Salesforce, JIRA, Zendesk). \nWorking knowledge of Amazon QuickSight, including dataset creation, calculated fields, parameters, and dashboard publishing. \nFamiliarity with columnar/lakehouse table formats such as Apache Iceberg, Delta Lake, or Apache Hudi. \nSolid understanding of data warehouse concepts: dimensional modeling, slowly changing dimensions, incremental vs. full loads. \nComfortable working independently with minimal day-to-day supervision, managing your own task priorities, and communicating status asynchronously. \nStrong written and verbal English communication skills; able to participate in cross-timezone meetings and write clear documentation. \nPreferred Qualifications\nExperience with infrastructure-as-code tools such as AWS CDK, CloudFormation, or Terraform. \nFamiliarity with CI/CD pipelines (GitHub Actions, CodePipeline) for data engineering workflows. \nExposure to PII detection/redaction practices or data governance frameworks. \nPrior experience translating Tableau workbooks or Excel-based reports into another BI platform. \nKnowledge of EventBridge-driven architectures and SQS/SNS messaging patterns. \nAWS certifications (e.g., Data Analytics Specialty, Solutions Architect Associate) are a plus. \nWhat We Offer\nThe opportunity to work on a greenfield BI platform with modern cloud-native tooling. \nA collaborative, small-team environment where your contributions have direct and visible impact. \nExposure to a broad technology stack spanning data engineering, analytics, and cloud infrastructure. \nMentorship and growth opportunities within a growing data organization.","description_format":"text","description_chars":4924,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"English","level":"Upper-Intermediate (B2)","optional":false}]},"benefits":["Growth opportunities"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Revenue Cycle & Medical Billing","Health Insurance & Benefits"],"lifecycle":[{"event":"open","at":"2026-09-13T12:10:00Z"},{"event":"close","at":"2026-10-26T20:04:06Z"},{"event":"reopen","at":"2026-11-06T05:07:54Z"}],"visa":[],"liveness":{"score":22,"band":"cold","label":"Long shot","p_open":1,"p_active":0.641,"p_room":0.35,"age_days":58,"expected_fill_days":20,"reasons":["conf:1","velocity","win:tail"],"computed_at":"2026-11-11T05:49:22Z"},"pay":null,"html_url":"https://alion.io/job/healthedge-senior-data-engineer-2","json_url":"https://alion.io/job/healthedge-senior-data-engineer-2.json","meta":{"generated_at":"2026-11-11T18:12:13Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":34879,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-12T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":58605},"rest":"https://alion.io/mcp/rest/get_company?id=58605"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhealthedge-senior-data-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhealthedge-senior-data-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhealthedge-senior-data-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/healthedge-senior-data-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhealthedge-senior-data-engineer-2"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":864070}},{"tool":"search_jobs","arguments":{"company":"healthedge.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":864070},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}