{"id":2915189,"url":"https://alion.io/job/techcarrot-data-engineer-4","title":"Data Engineer","company":{"id":89346,"name":"Techcarrot","domain":"techcarrot.ae","url":"https://alion.io/company/techcarrot","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Zoho Recruit","truth_index":{"grade":"B","score":75,"open_postings":3,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-22T06:10:10Z"}},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":14000,"max_usd":35000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":21},"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Azure DevOps","optional":false},{"name":"Azure SQL Database","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Dimensional Modeling","optional":true},{"name":"Power BI","optional":true},{"name":"Rest API","optional":true}],"status":"live","first_seen_at":"2026-08-31T00:00:00Z","employer_posted_date":"2026-08-31","last_verified_at":"2026-10-22T16:03:03Z","board_verified":true,"closed_at":null,"days_open":52,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":52},"description":"Independently understand business and technical requirements and translate them into scalable data engineering solutions. \nTake end-to-end ownership of assigned projects, modules and data pipelines with minimal supervision.\nDesign, develop, test and deploy robust end-to-end data pipelines on the Azure Data Platform.\nBuild scalable and reusable data ingestion and transformation frameworks using Azure Data Factory and Azure Databricks.\nDevelop data transformation and processing logic using Python, PySpark and SQL.\nWork with structured, semi-structured and unstructured data from databases, APIs, files and other enterprise data sources.\nDesign and implement batch and, where required, near-real-time data processing solutions.\nDevelop optimized Databricks workloads using Apache Spark and Delta Lake.\nPerform data analysis, profiling and validation to identify data quality, completeness and consistency issues.\nImplement appropriate data quality checks, error handling, logging and monitoring within data pipelines.\nTroubleshoot data pipeline failures, performance issues and data discrepancies independently.\nOptimize SQL queries, Spark jobs and data pipelines for performance and scalability.\nCreate reusable components and frameworks to improve development efficiency across projects.\nParticipate in solution design and technical discussions and provide recommendations on implementation approaches.\nImplement CI/CD processes for data engineering components using Azure DevOps.\nDeploy and manage ADF, Databricks and related Azure data components across development, test and production environments.\nPrepare appropriate technical design, data mapping and operational documentation.\nCollaborate with solution architects, analysts, BI developers, source-system teams and other engineering teams.\nSupport production deployments and troubleshoot post-production issues when required.\nRequirements\nStrong hands-on experience with:\nAzure Data Platform\nAzure Data Factory (ADF) - pipelines, datasets, linked services, triggers, parameterization and reusable frameworks \nAzure Databricks\nAzure Data Lake Storage Gen2 (ADLS Gen2)\nAzure Synapse Analytics / Azure SQL\nAzure Key Vault\nAzure Monitor / Log Analytics or equivalent monitoring capabilities\nAzure Event Hubs - exposure/experience is preferred\nDatabricks & Data Processing\nStrong hands-on experience with Databricks and Apache Spark \nStrong PySpark development skills\nExperience with Delta Lake / Delta tables\nUnderstanding of Medallion or similar layered data architectures\nExperience implementing incremental and full-load processing patterns\nAbility to troubleshoot and optimize Spark workloads\nExperience with Databricks Workflows/Jobs\nKnowledge of Unity Catalog is preferred\nSQL & Python\nStrong SQL development skills \nAbility to write and optimize complex SQL queries\nExperience with joins, CTEs, window functions and analytical queries\nStrong working knowledge of Python\nAbility to develop reusable Python/PySpark modules and utilities\nDevOps / CI-CD\nHands-on experience with Azure DevOps \nGit-based source control\nBranching and code-management practices\nCI/CD implementation for Azure Data Factory and Databricks\nExperience deploying solutions across multiple environments\nQualifications & Experience \n4-6 years of overall experience in Data Engineering / Data Platform development.\nPreferably 4+ years of hands-on experience with Microsoft Azure Data Platform technologies.\nStrong hands-on experience with Azure Data Factory, Databricks, SQL and Python/PySpark.\nBachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.\nExperience delivering at least one or more end-to-end Azure data engineering projects.\nGood understanding of data warehousing, data lake and modern data platform concepts.\nStrong analytical and problem-solving skills.\nAbility to work independently and manage multiple technical activities.\nGood written and verbal communication skills.\n Preferred Skills \nMicrosoft Azure / Databricks certifications.\nExperience with Unity Catalog and Databricks governance.\nExperience with REST API-based data ingestion.\nKnowledge of dimensional modelling and data warehouse concepts.\nExperience working with Power BI or downstream analytics platforms.\nExposure to streaming/event-driven data processing.\nExperience working in enterprise-scale Azure environments.","description_format":"text","description_chars":4374,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development","Enterprise Apps Implementation Partners"],"lifecycle":[{"event":"open","at":"2026-10-20T10:31:31Z"}],"visa":[],"liveness":{"score":29,"band":"fade","label":"Fading","p_open":1,"p_active":0.637,"p_room":0.45,"age_days":52,"expected_fill_days":30,"reasons":["conf:1","wave","velocity","win:tail"],"computed_at":"2026-10-22T06:10:10Z"},"pay":null,"html_url":"https://alion.io/job/techcarrot-data-engineer-4","json_url":"https://alion.io/job/techcarrot-data-engineer-4.json","meta":{"generated_at":"2026-10-22T16:03:20Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16158,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":89346},"rest":"https://alion.io/mcp/rest/get_company?id=89346"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftechcarrot-data-engineer-4"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftechcarrot-data-engineer-4"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftechcarrot-data-engineer-4"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/techcarrot-data-engineer-4\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftechcarrot-data-engineer-4"}]}