{"id":945659,"url":"https://alion.io/job/hg-insights-principal-data-engineer","title":"Principal Data Engineer","company":{"id":679293,"name":"HG Insights","domain":"hginsights.com","url":"https://alion.io/company/hginsights","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":28000,"max_usd":60000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":15,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"AI Agents","optional":false},{"name":"Amazon EC2","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"Java","optional":false},{"name":"MySQL","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Docker","optional":true},{"name":"GDPR","optional":true},{"name":"Kubernetes","optional":true},{"name":"Machine Learning","optional":true},{"name":"MLlib","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-09-09T08:29:36Z","employer_posted_date":"2026-09-09","last_verified_at":"2026-11-08T12:10:45Z","board_verified":true,"closed_at":null,"days_open":60,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":60},"description":"About HG\nHG Insights is the pioneer of Revenue Growth Intelligence. For more than a decade, we have delivered comprehensive, AI- driven datasets on B2B buyers, technology adoption, IT spend, and buyer intent, sourced from billions of data points.\nToday, we are a trusted partner to Fortune 500 technology companies, hyperscalers, and innovative B2B vendors seeking precise go-to-market analytics and decision-making. Through an evolving suite of AI agents that incorporate first-party data and buyer signals, HG Insights enables AI-powered GTM automation across sales, marketing, RevOps, and data analytics teams, modernizing GTM execution from strategy through activation.\nAbout the Role\nThis is a senior individual contributor role on the team that builds and runs HG's data platform: everything between a raw vendor file landing in our lake and a finished dataset arriving in a customer's hands. It includes the transformation stack that turns messy external signals into product-grade company attributes, the matching systems that decide which company a piece of evidence belongs to and the internal applications our analysts use to curate and correct the result.\nYou will be based in Pune, working with engineers at our Pune, US & Brazil locations.\nWhat you’ll do\nOwn the architecture of the data platform end to end, and make the calls on build vs. buy, batch vs. incremental, and where each workload belongs.\nBuild the quality layer that catches silent failure: contracts at every handoff, freshness and completeness monitoring, and pipelines that quarantine bad data rather than publish it.\nImprove entity matching and attribution - and make match decisions explainable to the customers who depend on them.\nMake our release cycle boring. Recurring deliveries should not depend on people watching them.\nTreat compute cost as a first-class engineering metric across Spark, orchestration, and the serving tier.\nWhat we are looking for\n15+ years building production grade data engineering systems, with a minimum of 5 years in a staff or principal scope.\nDeep Spark and Databricks expertise at multi-terabyte scale, including the operational side of lakehouse table formats - merges, compaction, small files, schema evolution.\nStrong SQL and dimensional modelling, and the judgment to know when to break the rules. \nExperience with performant and scalable OLTP setups (MySQL/Postgres).\nAirflow orchestration (DAGs, operators, sensors) and integration with Spark/Databricks.\nProven experience in AWS ecosystems (EC2, S3, EMR).\nHands-on entity resolution, fuzzy matching, or record linkage at scale. This is central to what we do.\nPython/Scala/Java, with real software discipline: testing, CI/CD, code review, infrastructure as code is a plus.\nExperience in Docker - Kubernetes, Terraform is a plus.\nExperience in integrating AI first implementations in traditional data engineering setups will greatly help in shaping our future designs.\nExperience with machine learning pipelines (Spark MLlib, Databricks ML) for predictive analytics.\nKnowledge of data governance frameworks and compliance standards (GDPR, CCPA).\nPreferred Qualifications\nB2B firmographic, technographic or intent data; streaming and CDC; warehouse-to-lakehouse migration experience.\nWhat we offer/Benefits\nWe take care of our people fully and genuinely. Every HG Insights employee receives comprehensive healthcare coverage for themselves and their families, a monthly wellness reimbursement and access to Able To professional mental health coaching for stress, anxiety and depression support as per the regions. For your financial future, US employees benefit from a 401(k) with up to 5% company matching and all UK employees are enrolled in a pension scheme. Every employee also receives stock options because the people who build this company should share in its success.\nWe invest in your growth through an annual learning reimbursement and a genuine commitment to developing your career at every stage. And through our Culture Club, you will always have a real voice in shaping the kind of workplace we build together.\nEqual Opportunity Employer\nHG Insights is proud to be an Equal Employment Opportunity employer. We do not accept unsolicited resumes from recruiters or employment agencies. In the event of a recruiter or agency submitting a resume or candidate without a signed agreement being in place, we explicitly reserve the right to pursue and hire such candidates without any financial obligation to the recruiter or agency. Any unsolicited resumes, including those submitted directly to hiring managers, are deemed to be the property of HG Insights.","description_format":"text","description_chars":4636,"description_truncated":false,"requirements":{"experience_years_min":15,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Stock options"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"Brazil","iso":"BR","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Data & Analytics","Sales & Marketing","Lead Generation"],"lifecycle":[{"event":"open","at":"2026-09-15T20:36:25Z"}],"visa":[],"liveness":{"score":28,"band":"fade","label":"Fading","p_open":1,"p_active":0.628,"p_room":0.45,"age_days":59,"expected_fill_days":31,"reasons":["conf:1","win:tail"],"computed_at":"2026-11-08T05:47:12Z"},"pay":null,"html_url":"https://alion.io/job/hg-insights-principal-data-engineer","json_url":"https://alion.io/job/hg-insights-principal-data-engineer.json","meta":{"generated_at":"2026-11-08T16:05:15Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":33939,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-09T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":679293},"rest":"https://alion.io/mcp/rest/get_company?id=679293"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhg-insights-principal-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhg-insights-principal-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhg-insights-principal-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/hg-insights-principal-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhg-insights-principal-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":945659}},{"tool":"search_jobs","arguments":{"company":"hginsights.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":945659},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}