{"id":3290334,"url":"https://alion.io/job/irth-data-engineer-mid-level-orbit","title":"Data Engineer (Mid Level)-Orbit","company":{"id":679084,"name":"IRTH","domain":"irth.ae","url":"https://alion.io/company/irth","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":{"grade":"B","score":75,"open_postings":3,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":38,"computed_at":"2026-10-27T05:46:45Z"}},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["IN"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":68000,"max_usd":162000,"period":"year","method":"global_role_seniority_cell","sample_n":1015},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Platform Engineering","optional":false},{"name":"pySpark","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Airflow","optional":true},{"name":"AWS Glue","optional":true},{"name":"Git","optional":true},{"name":"Power BI","optional":true},{"name":"Python","optional":true}],"status":"live","first_seen_at":"2026-09-13T07:02:46Z","employer_posted_date":null,"last_verified_at":"2026-09-13T07:02:46Z","board_verified":false,"closed_at":null,"days_open":44,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":44},"description":"About Irth SolutionsIrth Solutions is a leading provider of cloud-based SaaS software for damage prevention, asset integrity, stakeholder engagement and land management, helping energy, utility, telecom, and infrastructure companies protect their critical network infrastructure. With nearly three decades of industry experience, Irth serves customers across North America and continues to expand its platform with new data-driven and AI-powered capabilities.\nData Engineer – Insights (AI/ML)Location: Remote – India\nDepartment: Insights (AI/ML)\nReports to: Data Platform & Analytics Manager\nAbout the RoleIrth is building a modern, multi-cloud, enterprise-grade data estate—a unified Databricks-based data platform that centralizes data across Irth’s products and cloud environments, including AWS, Azure, and GCP.\nAs a Data Engineer, you will play a hands-on implementation role, working closely with the Senior Data Architect to bring the enterprise data platform vision to life.\nYou will design and develop data pipelines based on established architectural patterns, implement data quality and governance controls, build Delta Lake and medallion architecture solutions, and help operationalize the new data platform.\nThis is an excellent opportunity for a mid-level Data Engineer looking to deepen their expertise in Databricks, Apache Spark, cloud data engineering, and modern lakehouse architecture while working in a multi-cloud enterprise environment.\nKey Responsibilities1. Data Pipeline Development – Primary Responsibility\nBuild, maintain, and enhance data ingestion pipelines across AWS, Azure, and GCP, following architecture and engineering patterns established by the Senior Data Architect.\n\nDevelop both batch and streaming pipelines using:\n\nDatabricks Workflows\n\nApache Spark / PySpark\n\nSQL\n\nDelta Live Tables\n\nDatabricks Lakeflow components\n\nImplement Bronze → Silver → Gold medallion architecture patterns for ingestion, transformation, cleansing, and standardization.\n\nImplement Change Data Capture (CDC) and Slowly Changing Dimensions (SCD Type 1 and Type 2).\n\nHandle schema evolution and changing source-system structures.\n\nImplement data validation, reconciliation, and quality rules as part of pipeline processing.\n\nBuild reusable and maintainable pipeline components following established engineering standards.\n\n2. Platform & Storage Implementation\nConfigure and maintain Delta Lake storage structures, tables, schemas, partitions, and optimization routines.\n\nApply Delta Lake performance and maintenance practices, including:\n\nOPTIMIZE\n\nZ-ORDER\n\nVACUUM\n\nAppropriate partitioning and file-management strategies\n\nAssist with implementation of metadata, cataloging, and lineage standards using Unity Catalog.\n\nSupport integration between cloud storage platforms and Databricks, including:\n\nAmazon S3 → Databricks\n\nAzure Storage → Databricks\n\nGoogle Cloud Storage → Databricks\n\nAssist with implementation of scalable storage and processing patterns defined by the Data Architect.\n\n3. Data Governance, Quality & Compliance Enablement\nImplement automated data-quality checks, profiling, validation, and monitoring in accordance with enterprise governance standards.\n\nApply data-quality rules at appropriate stages of the Bronze, Silver, and Gold layers.\n\nImplement RBAC policies, security controls, and data-classification tags defined by the enterprise governance model.\n\nSupport implementation of metadata and lineage mapping across Unity Catalog and Microsoft Purview.\n\nHelp ensure datasets are properly documented, classified, governed, and discoverable.\n\nSupport remediation of data-quality and governance issues identified through monitoring or reviews.\n\n4. Orchestration, Automation & Operational Support\nBuild, schedule, monitor, and maintain production workflows using:\n\nDatabricks Workflows\n\nDelta Live Tables\n\nAzure Data Factory (ADF)\n\nOther approved orchestration tools\n\nContribute to CI/CD pipelines for data-engineering code, including source control, automated testing, deployment, and environment management.\n\nSupport DEV → QA → PROD promotion processes.\n\nMonitor production pipelines and respond to failures and data-quality issues.\n\nTroubleshoot failed jobs, investigate root causes, and support pipeline recovery.\n\nPerform performance tuning across Spark jobs, SQL workloads, Delta tables, and data pipelines.\n\nParticipate in operational improvements that increase pipeline reliability, scalability, and cost efficiency.\n\n5. Collaboration & Documentation\nWork directly with the Senior Data Architect to translate architecture designs and technical standards into actionable implementation tasks.\n\nParticipate in architecture reviews, technical design discussions, coding reviews, and engineering standards meetings.\n\nCollaborate with Data Scientists, ML Engineers, Analysts, Product teams, and other engineering stakeholders to understand data requirements.\n\nDocument:\n\nData pipelines\n\nData flows\n\nData dictionaries\n\nTransformation logic\n\nData-quality rules\n\nTest cases\n\nJob schedules\n\nOperational procedures\n\nMaintain clear and accurate technical documentation to support platform adoption, troubleshooting, and future development.\n\nProvide implementation feedback to the Data Architect and identify opportunities to improve platform patterns, tooling, and developer experience.\n\nRole ScopeThis is primarily an implementation-focused Data Engineering role. The Senior Data Architect will establish the overall platform architecture, standards, and design patterns; the Data Engineer will translate those patterns into reliable, production-ready pipelines and platform capabilities.\nThe role provides an opportunity to gain deeper hands-on experience with Databricks, Spark, Delta Lake, Unity Catalog, cloud data platforms, data governance, and multi-cloud lakehouse engineering while contributing to a strategic enterprise data platform.\nRequirementsQualificationsRequired Qualifications\n\n3–5 years of experience in Data Engineering, ETL development, or cloud data platform engineering.\n\nHands-on experience with Databricks, Apache Spark, PySpark, or other distributed data-processing technologies.\n\nStrong proficiency in SQL, including structured data transformation, joins, aggregations, and performance-aware query development.\n\nExperience working with at least one major cloud platform, with Microsoft Azure preferred; AWS and/or GCP experience is also valuable.\n\nUnderstanding of core data-engineering concepts, including:\n\nData modeling\n\nData quality\n\nSchema evolution\n\nData validation\n\nPipeline monitoring and troubleshooting\n\nBasic understanding of data-security practices, including:\n\nRole-Based Access Control (RBAC)\n\nEncryption\n\nCredential and secret management\n\nSecure access to cloud and data-platform resources\n\nPreferred Qualifications\nHands-on or working knowledge of Delta Lake, medallion architecture, and modern lakehouse best practices.\n\nExperience with metadata, cataloging, and governance platforms such as:\n\nUnity Catalog\n\nMicrosoft Purview\n\nAWS Glue Data Catalog\n\nSimilar enterprise metadata and data-governance tools\n\nExperience with workflow orchestration and scheduling technologies, such as:\n\nAzure Data Factory (ADF)\n\nDatabricks Workflows\n\nApache Airflow\n\nDatabricks Jobs / DBX\n\nSimilar orchestration frameworks\n\nExperience with Git-based development, CI/CD, and DevOps practices.\n\nKnowledge or experience in one or more of the following areas:\n\nGeospatial/GIS data\n\nBI semantic layers, particularly Power BI\n\nData preparation for AI/ML workloads\n\nRelevant cloud or Databricks certifications, such as:\n\nDatabricks Data Engineer Associate\n\nMicrosoft Azure Data Engineer Associate (DP-203)\n\nEquivalent cloud or data-engineering certifications\n\nNice-to-Have Qualifications\nUnderstanding of asset integrity management concepts, including inspection data, risk scoring, corrosion tracking, defect management, and maintenance data as applied to pipeline or utility operations.\n\nPrevious experience working with or integrating oil & gas, utility, infrastructure, or pipeline asset data into enterprise data platforms.\n\nExperience working with:\n\nPipeline and facility data\n\nGIS/geospatial asset data\n\nInspection and maintenance records\n\nAsset-risk datasets\n\nFamiliarity with regulatory, compliance, and audit-reporting requirements associated with pipeline, utility, or asset-integrity data.\n\nSuccess MetricsSuccess in this role will be measured by the engineer’s ability to reliably implement and operationalize the data-platform patterns established by the Data Architect.\nKey measures include:\n\nHigh-quality implementation of ingestion, transformation, data-quality, and governance patterns defined by the Data Architect.\n\nReliable and maintainable pipelines supporting consistent Bronze → Silver → Gold data flows.\n\nStrong adherence to cataloging, metadata, lineage, security, and data-governance standards.\n\nReduction in pipeline failures and production incidents through improved monitoring, testing, troubleshooting, and operational practices.\n\nContinuous improvements in pipeline performance, scalability, reliability, and maintainability.\n\nClear and complete technical documentation covering pipelines, transformations, data-quality rules, and operational procedures.\n\nEffective collaboration with the Senior Data Architect, engineering teams, product teams, and business stakeholders.\n\nDemonstrated ability to take architecture guidance and translate it into production-ready, scalable data-engineering solutions.\n\nBenefits\nBenefits\n\nCompetitive Salary – A competitive compensation package based on experience and qualifications.\n\nMedical, Dental, and Vision Insurance – Comprehensive insurance coverage to support you and your family.\n\n401(k) Plan with Company Match.\n\nGenerous Paid Time Off (PTO) – Time off to support work-life balance and personal needs.\n\nCompany-Paid Holidays – Paid holidays throughout the year.\n\nFlexible Work Options – Work-from-home opportunities are available, depending on role and business needs.\n\nOn-Call Compensation – Additional pay for eligible on-call shifts.\n\nOriginally posted on Himalayas","description_format":"text","description_chars":10097,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Flexible schedule","Insurance coverage","Vision insurance"],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Property Development"],"lifecycle":[{"event":"open","at":"2026-10-26T19:11:04Z"}],"visa":[],"liveness":{"score":25,"band":"fade","label":"Fading","p_open":0.6,"p_active":0.772,"p_room":0.55,"age_days":43,"expected_fill_days":38,"reasons":["seen:43","win:tail"],"computed_at":"2026-10-27T05:46:45Z"},"pay":null,"html_url":"https://alion.io/job/irth-data-engineer-mid-level-orbit","json_url":"https://alion.io/job/irth-data-engineer-mid-level-orbit.json","meta":{"generated_at":"2026-10-28T02:47:47Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3719,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-29T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":679084},"rest":"https://alion.io/mcp/rest/get_company?id=679084"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Firth-data-engineer-mid-level-orbit"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Firth-data-engineer-mid-level-orbit"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Firth-data-engineer-mid-level-orbit"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/irth-data-engineer-mid-level-orbit\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Firth-data-engineer-mid-level-orbit"}]}