{"id":37406,"url":"https://alion.io/job/socotec-senior-data-engineer-azure-databricks","title":"Senior Data Engineer (Azure & Databricks)","company":{"id":7210,"name":"SOCOTEC","domain":"socotec.com","url":"https://alion.io/company/socotec","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"SmartRecruiters","truth_index":{"grade":"B","score":73,"open_postings":40,"ghost_share":0,"stale_share":0.9,"repost_share":0,"time_to_fill_p50_days":65,"computed_at":"2026-10-17T05:46:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["New York, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":149000,"max":190000,"currency":"USD","period":"year","gross":null,"usd_annual":190000},"salary_estimate":null,"experience_years_min":7,"visa_sponsorship":true,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Machine Learning","optional":false},{"name":"Master Data Management","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"AWS","optional":true},{"name":"Azure","optional":true},{"name":"GCP","optional":true}],"status":"live","first_seen_at":"2026-08-03T18:15:09Z","employer_posted_date":"2026-08-03","last_verified_at":"2026-10-17T12:22:44Z","board_verified":true,"closed_at":null,"days_open":74,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":74},"description":"SOCOTEC is a provider of niche consulting services within the architecture, engineering and construction industries focusing on high-performance buildings and specialty structures. Through an integrated, holistic approach, our professionals provide solutions for building envelope, energy efficiency, sustainability, code advisory and construction advisory projects\nSOCOTEC’s combination of expertise, experience and methods are increasingly recognized as best-in-class fueling significant growth.\nSOCOTEC is proud to announce that it has been officially certified as a Great Place to Work by the renowned global authority on workplace culture, Great Place to Work®.\n As a Senior Data Engineer, you will design and build scalable data systems that power SOCOTEC’s digital platforms and enterprise analytics. You will play a key role in architecting our modern data stack, enabling reliable ingestion, transformation, and modeling of data across multiple enterprise systems.\nThis role requires strong experience in Databricks, distributed data processing, and enterprise data architecture, along with the ability to lead large data initiatives and collaborate with engineering and business teams.\nYou will work closely with software engineers and platform teams to ensure SOCOTEC’s data ecosystem is reliable, scalable, and optimized for both operational systems and advanced analytics.\nKey Responsibilities\nBuild SOCOTEC’s Enterprise Data Platform\nDesign and implement scalable data pipelines that ingest and transform data from enterprise systems such as ERP, CRM, and operational databases.\nDevelop and maintain data pipelines in Databricks using Spark and Delta Lake.\nBuild and maintain data models that support analytics, AI applications, and operational systems.\nLead Master Data Management (MDM) Initiatives\nArchitect and implement SOCOTEC’s custom MDM platform using Databricks.\nDesign data models that establish consistent “golden records” across multiple enterprise systems.\nImplement data governance, lineage, and quality frameworks.\nDesign Enterprise-Scale Data Pipelines\nBuild reliable ingestion pipelines for large-scale structured and semi-structured data.\nImplement batch and near-real-time data processing pipelines.\nOptimize large-scale data processing jobs for performance and cost efficiency.\nEnable Data for AI and Advanced Analytics\nPartner with AI and software engineering teams to deliver high-quality datasets for machine learning and AI applications.\nBuild data pipelines that support SOCOTEC’s digital products, analytics dashboards, and operational platforms.\nDrive Data Architecture and Best Practices\nDefine standards for data modeling, pipeline design, and data quality.\nImplement monitoring, observability, and alerting data systems.\nEnsure scalability, reliability, and security across the data platform.\n 7-10 years with hands on experience with Databricks\nExperience designing and owning large enterprise data projects\nStrong hands-on experience with Databricks, Apache Spark, and Delta Lake\nExperience building enterprise-scale ETL/ELT pipelines\nExperience working in enterprise environments with complex data systems\nStrong SQL and Python skills\nPreferred Experience\nExperience building Master Data Management (MDM) systems\nExperience with modern cloud data stacks (AWS, Azure, or GCP)\nExperience integrating data from ERP systems such as Deltek, SAP, or Oracle\nExperience with data governance, data lineage, and metadata management\nFamiliarity with lakehouse architectures and modern data platforms\nTechnical Skills\nDatabricks\nApache Spark\nDelta Lake\nPython\nSQL\nData modeling\nETL / ELT pipeline development\nCloud data platforms (AWS / Azure)\nData architecture and distributed systems\nWe are looking for someone who:\nHas owned and delivered large-scale data platforms\nIs comfortable working in complex enterprise environments\nCan design systems that scale across multiple business units and data sources\nIs passionate about building reliable data infrastructure that enables analytics and AI\nThis role requires both deep technical expertise and architectural thinking, as you will play a key role in shaping SOCOTEC’s long-term data strategy.\nWhy Join SOCOTEC Digital Assurance\nYou will be joining a growing technology group building advanced platforms for the built environment, including:\nAI-powered compliance systems\nIntelligent inspection workflows\nEnterprise analytics platforms\nLarge-scale data infrastructure supporting engineering services\nOur goal is to transform how data and expertise flow across the built environment, enabling smarter decisions, faster compliance, and more efficient infrastructure.\nIf you are passionate about building modern data platforms and solving complex data challenges at scale, we’d love to hear from you.\n Additional information\nFor more information, please visit www.socotec.us\nJob Type: Full-Time, 4 days a week in office with one hybrid/ remote day\nSalary: $149-190k\nApplicants must be legally authorized to work in the United States without the need for current or future employer sponsorship.\nAll your information will be kept confidential according to EEO guidelines.\nSOCOTEC is an Equal Opportunity Employer.","description_format":"text","description_chars":5193,"description_truncated":false,"requirements":{"experience_years_min":7,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"English","level":"All levels","optional":false}]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Testing, Inspection & Certification"],"lifecycle":[{"event":"open","at":"2026-08-03T18:15:09Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 2","filings_12m":2,"filings_prev_12m":0,"green_card_filings_12m":0,"median_offered_wage_usd":77500,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":0}],"liveness":{"score":22,"band":"cold","label":"Long shot","p_open":1,"p_active":0.505,"p_room":0.44,"age_days":74,"expected_fill_days":65,"reasons":["conf:3","stale_co","win:tail","crowd:brand"],"computed_at":"2026-10-17T05:46:00Z"},"pay":{"stated_usd_annual":190000,"is_top_pay":true},"html_url":"https://alion.io/job/socotec-senior-data-engineer-azure-databricks","json_url":"https://alion.io/job/socotec-senior-data-engineer-azure-databricks.json","meta":{"generated_at":"2026-10-17T14:42:47Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":19809,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":7210},"rest":"https://alion.io/mcp/rest/get_company?id=7210"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsocotec-senior-data-engineer-azure-databricks"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsocotec-senior-data-engineer-azure-databricks"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsocotec-senior-data-engineer-azure-databricks"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/socotec-senior-data-engineer-azure-databricks\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsocotec-senior-data-engineer-azure-databricks"}]}