{"id":4309742,"url":"https://alion.io/job/newvision-software-consultancy-senior-software-engineer-data","title":"Senior Software Engineer- Data","company":{"id":4489094,"name":"NewVision Software & Consultancy","domain":"newvision-software.com","url":"https://alion.io/company/newvision-software-and-consultancy","size_band":"501-1000","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Career site","truth_index":{"grade":"B","score":75,"open_postings":36,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-17T05:47:56Z"}},"role":"Backend","role_family":"Backend","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Rest API","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-10-21T00:00:00Z","employer_posted_date":"2026-11-14","last_verified_at":"2026-11-17T13:35:28Z","board_verified":true,"closed_at":null,"days_open":27,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":27},"description":"The earlier req for this position was closed with -ve BGV candidate. Thus, opening a new one.\nRole: Senior Azure Databricks Developer\nRole Summary:\nSenior Databricks Engineer with strong expertise in developing, optimizing, and maintaining data engineering solutions on the Databricks platform. The candidate should have deep experience with Apache Spark (PySpark), Databricks notebooks, Delta Lake, and Spark performance optimization. This role is focused on Spark-native data engineering rather than ETL tool development.\nMust Have Skills:\nStrong hands-on experience with Databricks notebooks, Jobs, Workflows, Repos, and Unity Catalog\nStrong expertise in PySpark and Spark SQL\nExperience building scalable Spark-based data pipelines\nStrong understanding of Spark DataFrames, joins, aggregations, window functions, and distributed processing\nExperience with Delta Lake (MERGE, OPTIMIZE, ZORDER, VACUUM, Change Data Feed)\nHands-on experience in Spark job performance tuning\nExperience optimizing partitions, shuffles, caching, memory usage, and cluster utilization\nExperience developing batch and incremental data processing pipelines\nStrong programming skills in Python, PySpark, and SQL\nExperience working on Azure Databricks, AWS Databricks, or GCP Databricks\nExperience with Git, CI/CD, code reviews, testing, and production deployments\nImplement event-driven data flows and real-time data ingestion using Apache Kafka.\nDevelop and integrate REST APIs for both data extraction and downstream data serving.\nMonitor, tune, and optimize PySpark workloads for maximum performance and cost efficiency.\nCommunicate effectively with cross-functional stakeholders to gather requirements and translate them into scalable technical solutions.\nMentor junior developers, guide architectural decisions, and prepare to take on team lead responsibilities in the future. as nice to have.\nGood to Have Skills:\nExperience with Lakehouse and Medallion architecture\nKnowledge of Data Modeling and Data Governance\nNice to Have Skills:\nExperience with Delta Live Tables (DLT)\nExperience mentoring teams or leading technical discussions\nExperience with Photon and Databricks Asset Bundles\nExperience with Cluster Policies and Workspace Administration\nDesign, build, and maintain robust batch and streaming data pipelines using Databricks and PySpark.\nExtract complex data from a variety of upstream sources (databases, internal/external APIs, flat files) and process it into the data lake.\nArchitect and optimize Delta Tables, utilizing the Medallion architecture (Bronze, Silver, Gold layers) for structured data governance.\nSoft skills:\nMust have strong written and verbal communication skills, excellent articulation skills, and experience working with client teams. \nThe candidate should have actively participated in and contributed to sprint planning and solution design.","description_format":"text","description_chars":2859,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-14T11:21:27Z"}],"visa":[],"liveness":{"score":56,"band":"ok","label":"Likely open","p_open":1,"p_active":0.625,"p_room":0.9,"age_days":27,"expected_fill_days":40,"reasons":["conf:2","stale_co","velocity","win:mid","comp:brand"],"computed_at":"2026-11-17T05:47:56Z"},"pay":null,"html_url":"https://alion.io/job/newvision-software-consultancy-senior-software-engineer-data","json_url":"https://alion.io/job/newvision-software-consultancy-senior-software-engineer-data.json","meta":{"generated_at":"2026-11-17T15:20:32Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":26927,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4489094},"rest":"https://alion.io/mcp/rest/get_company?id=4489094"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnewvision-software-consultancy-senior-software-engineer-data"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnewvision-software-consultancy-senior-software-engineer-data"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnewvision-software-consultancy-senior-software-engineer-data"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/newvision-software-consultancy-senior-software-engineer-data\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnewvision-software-consultancy-senior-software-engineer-data"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4309742}},{"tool":"search_jobs","arguments":{"company":"newvision-software.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4309742},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}