{"id":3481844,"url":"https://alion.io/job/allstate-data-engineer-consultant-ii","title":"Data Engineer Consultant II","company":{"id":4796,"name":"Allstate","domain":"allstate.com","url":"https://alion.io/company/allstate","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Paylocity","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":14500,"max_usd":38000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":21},"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"CI/CD","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Machine Learning","optional":false},{"name":"Microsoft Fabric","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-10-29T17:27:56Z","employer_posted_date":null,"last_verified_at":"2026-10-29T17:27:56Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"At Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. And for more than 90 years, our innovative drive has kept us a step ahead of our customers evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection.\n\nJob Description\n\nAllstate's Data & Analytics Technology organization is seeking a Data Engineer to design, build, and operate scalable, reliable, and high performing data pipelines that support enterprise analytics, reporting, and advanced data use cases. In this role, you will focus on building robust batch and streaming data solutions using Apache Spark and modern cloud data platforms. Experience with Microsoft Fabric is a strong plus.\n\nYou will work closely with analytics engineers, data scientists, product teams, and platform partners to transform raw, complex data into trusted, analytics ready datasets. This role plays a critical part in enabling data driven decision making by ensuring data quality, performance, scalability, and operational excellence across the data platform.\n\nKey Responsibilities\n\nDesign, build, and maintain scalable batch and streaming data pipelines using Apache Spark and cloud‑native data technologies.\nDevelop and optimize ETL/ELT workflows to ingest, transform, and curate data from diverse source systems into analytics‑ready datasets.\nImplement data modeling and transformation logic to support reporting, dashboards, and downstream analytical and machine learning workloads.\nBuild and manage data processing workloads within modern Lakehouse platforms, including Microsoft Fabric / One Lake (preferred).\nEnsure data quality, reliability, and consistency by implementing validation checks, monitoring, and reconciliation processes.\nOptimize Spark jobs for performance, cost efficiency, and scalability across large and complex datasets.\nManage and evolve data schemas while handling schema drift and upstream source changes.\nDevelop reusable frameworks, libraries, and standardized patterns to improve data engineering productivity and consistency.\nImplement CI/CD pipelines for data workloads to enable automated testing, deployment, and rollback.\nMonitor data pipelines and jobs, troubleshoot failures, and resolve performance or data quality issues.\nPartner with analytics engineers, BI developers, and data scientists to understand data requirements and deliver curated datasets.\nCollaborate with platform, security, and governance teams to ensure data security, compliance, and proper access controls.\nContribute to Agile delivery processes, including sprint planning, design reviews, and continuous improvement initiatives.\n\nRequired Qualifications\n\nStrong experience as a Data Engineer building and operating production data pipelines.\nHands‑on experience with Apache Spark for large‑scale data processing.\nProficiency in Python, SQL, and data transformation best practices.\nExperience with cloud‑based data platforms and storage (e.g., Data Lakes, Lakehouse architectures).\nFamiliarity with Microsoft Fabric, One Lake, or similar analytics platforms (strong plus).\nExperience designing and optimizing data models for analytical workloads.\nUnderstanding of distributed data processing concepts, performance tuning, and fault tolerance.\nExperience with CI/CD, version control, and infrastructure‑as‑code concepts.\nStrong problem‑solving skills and ability to troubleshoot complex data issues.\nExcellent communication skills and ability to collaborate across technical and non‑technical teams.\n4+ years of experience in data engineering or equivalent role (preferred).\n\nPreferred / Nice‑to‑Have Skills\n\nExperience with real‑time or event‑driven data processing.\nFamiliarity with data governance, metadata management, and data quality frameworks.\nExposure to orchestration tools and workflow management systems.\nExperience supporting analytical, reporting, or machine learning use cases.\n\nSupervisory Responsibilities\n\nThis job does not have supervisory responsibilities.\n\nPrimary Skills\n\nShift Time\n\nRecruiter Info\n\nYateesh B G\n\n[HIDDEN TEXT]\n\nAbout Allstate\n\nJoining our team isn't just a job — it's an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. One where you can shape the future of protection while supporting causes that mean the most to you. Joining our team means being part of something bigger – a winning team making a meaningful impact.\n\nThe Allstate Corporation is one of the largest publicly held insurance providers in the United States. Ranked No. 84 in the 2023 Fortune 500 list of the largest United States corporations by total revenue, The Allstate Corporation owns and operates 18 companies in the United States, Canada, Northern Ireland, and India. Allstate India Private Limited, also known as Allstate India, is a subsidiary of The Allstate Corporation. The India talent center was set up in 2012 and operates under the corporation's Good Hands promise. As it innovates operations and technology, Allstate India has evolved beyond its technology functions to be the critical strategic business services arm of the corporation. With offices in Bengaluru and Pune, the company offers expertise to the parent organization's business areas including technology and innovation, accounting and imaging services, policy administration, transformation solution design and support services, transformation of property liability service design, global operations and integration, and training and transition.\n\nLearn more about Allstate India here.","description_format":"text","description_chars":5719,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Property & Casualty Insurance"],"lifecycle":[{"event":"open","at":"2026-10-29T20:10:15Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":26,"reasons":["seen:1","velocity","win:early"],"computed_at":"2026-10-31T05:58:55Z"},"pay":null,"html_url":"https://alion.io/job/allstate-data-engineer-consultant-ii","json_url":"https://alion.io/job/allstate-data-engineer-consultant-ii.json","meta":{"generated_at":"2026-10-31T11:40:03Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":17955,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-01T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4796},"rest":"https://alion.io/mcp/rest/get_company?id=4796"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fallstate-data-engineer-consultant-ii"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fallstate-data-engineer-consultant-ii"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fallstate-data-engineer-consultant-ii"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/allstate-data-engineer-consultant-ii\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fallstate-data-engineer-consultant-ii"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3481844}},{"tool":"search_jobs","arguments":{"company":"allstate.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3481844},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}