{"id":3822668,"url":"https://alion.io/job/7elevengsc-software-engineer-ii-data-engineering","title":"Software Engineer II (Data Engineering)","company":{"id":1047705,"name":"7Elevengsc","domain":"7-elevengsc.com","url":"https://alion.io/company/7elevengsc","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"iCIMS","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Databricks","optional":false},{"name":"Java","optional":false},{"name":"Power BI","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"AI Agents","optional":true},{"name":"Amazon Kinesis","optional":true},{"name":"Apache Kafka","optional":true},{"name":"Docker","optional":true},{"name":"FastAPI","optional":true},{"name":"Git","optional":true},{"name":"Kubernetes","optional":true},{"name":"MySQL","optional":true},{"name":"PostgreSQL","optional":true},{"name":"RabbitMQ","optional":true}],"status":"live","first_seen_at":"2026-11-05T17:35:00Z","employer_posted_date":null,"last_verified_at":"2026-11-05T17:35:00Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"Responsibilities:\nDesign, develop, and maintain scalable data processing pipelines using the Spark ecosystem, including PySpark, Databricks, and Synapse Notebooks.\nDevelop and manage data pipelines using Azure Data Factory, Azure Synapse, and Azure Data Lake to ingest, process, and integrate data from Azure and on-premises sources.\nImplement full and incremental data load strategies, ensuring efficient, reliable, and timely movement of data across source systems and target platforms.\nPerform data cleansing, transformation, aggregation, and modeling to prepare high-quality datasets for analytics, reporting, and downstream consumption.\nApply Azure security, access control, and data governance practices to ensure secure and compliant data processing and storage.\nOptimize Spark jobs, data pipelines, and SQL workloads for performance, scalability, reliability, and cost efficiency.\nCollaborate with BI and analytics teams to support Power BI reports and dashboards, including datasets/models hosted on Azure Analysis Services (AAS) and Power BI.\nTroubleshoot data pipeline and processing issues, perform root-cause analysis, and implement appropriate fixes to maintain data availability and quality.\nDevelop reusable frameworks, processes, and best practices for data ingestion, transformation, validation, and monitoring.\nWork closely with business stakeholders, data analysts, and engineering teams to translate data requirements into scalable and maintainable data solutions.\n\nKey Skills Required:\n\nData Engineering & Analytics:\n5+ years of experience in statistics and analytics on large-scale datasets.\nStrong SQL and data analysis skills, including SQL DW concepts.\nExperience with Spark ecosystem: PySpark, Databricks, Synapse Notebooks (1+ year hands-on).\nProficiency in Azure Data Services:Azure Synapse\nAzure Data Factory\nAzure Data Lake\nAzure security model\n\nExperience building pipelines for full and incremental data loads from Azure and on-prem sources.\nStrong knowledge of data management, processing, cleaning, transformation, aggregation, and modeling.\nWorking knowledge of Power BI reports and dashboards (AAS / PBI).\n\nSystems, Cloud & DevOps:\nStrong programming skills in Python (Java is a plus).\nExperience with FastAPI, RESTful services, and microservices architecture.\nProficiency with databases such as MySQL, PostgreSQL, and MongoDB.\nExperience deploying applications using Docker and Kubernetes.\nExperience with streaming and messaging systems: Kafka, Kinesis, RabbitMQ.\nFamiliarity with Azure Git and DevOps practices.\n\nLeadership & Soft Skills:\nProven experience leading projects end-to-end.\nStrong mentoring and team leadership experience.\nExcellent written and verbal communication skills.\nAbility to work effectively in distributed teams.\nStrong understanding of data structures and design patterns.\nExposure to the retail industry is a plus.\n\nGood To Have Skills:\nAI / ML and Agentic AI.\n\nEducation:\nBachelor’s degree in computer science or a related field (required).\nMaster’s degree is a plus.\nSkills\nADF, Azure Data Factory, Azure Data Lake, Azure Synapse, Databricks, PySpark, Python, SQL, Spark, Synapse","description_format":"text","description_chars":3146,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Retail & Shop Systems","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-05T17:35:00Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":26,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/7elevengsc-software-engineer-ii-data-engineering","json_url":"https://alion.io/job/7elevengsc-software-engineer-ii-data-engineering.json","meta":{"generated_at":"2026-11-07T12:12:08Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13355,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1047705},"rest":"https://alion.io/mcp/rest/get_company?id=1047705"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2F7elevengsc-software-engineer-ii-data-engineering"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2F7elevengsc-software-engineer-ii-data-engineering"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2F7elevengsc-software-engineer-ii-data-engineering"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/7elevengsc-software-engineer-ii-data-engineering\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2F7elevengsc-software-engineer-ii-data-engineering"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3822668}},{"tool":"search_jobs","arguments":{"company":"7-elevengsc.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3822668},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}