{"id":1947476,"url":"https://alion.io/job/state-street-lead-data-engineer-assistant-vice-president","title":"Lead Data Engineer - Assistant Vice President","company":{"id":1806,"name":"State Street","domain":"statestreet.com","url":"https://alion.io/company/state-street","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":97,"open_postings":365,"ghost_share":0.014,"stale_share":0.274,"repost_share":0.049,"time_to_fill_p50_days":20,"computed_at":"2026-10-22T06:10:10Z"}},"role":"Data Science","role_family":"Data Science","seniority":"head","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":19000,"max_usd":47000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":99},"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"ETL/ELT","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-10-06T07:49:20Z","employer_posted_date":"2026-10-06","last_verified_at":"2026-10-22T03:57:21Z","board_verified":true,"closed_at":null,"days_open":15,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":15},"description":"Lead Data Engineer - Databricks, Python, PySpark & AWS\nJob Summary\nWe are seeking a Senior Data Engineer / SME to join the SSIM Business Architecture vertical in a hands-on Individual Contributor (IC) role.\nThe role requires strong expertise in Databricks, Python, PySpark, AWS, and SQL to design, build, and support scalable, production-grade data pipelines and cloud data solutions. Snowflake and enterprise data integration experience are desired.\nThe successful candidate will demonstrate strong technical ownership, problem-solving skills, and engineering best practices across the data ecosystem.\nRole location : Bangalore\nBusiness Architecture: The role sits within the SSIM Business Architecture vertical and requires close collaboration with Architecture, Security, engineering, platform, and business technology stakeholders.\nHybrid working: The role requires 4 days per week working from the office, in line with the applicable workplace expectations.\nWorking hours: The primary working window will be 12:00 PM to 9:00 PM India time, providing alignment with US stakeholders and teams.\nFlexibility: Given the enterprise and production nature of the role, the individual must be flexible to work outside standard hours and, where required, over weekends to support critical deliveries, production issues, releases,DR or other business needs.\nKey Responsibilities\nDesign, develop, and maintain scalable ETL/ELT, data ingestion, and transformation pipelines.\nBuild and optimize data processing solutions using Python and PySpark.\nDevelop and support production-grade workloads and pipelines on Databricks.\nDesign and implement cloud-native data engineering solutions on AWS.\nWork with large-scale structured and unstructured datasets.\nOptimize Spark and Databricks workloads for performance, scalability, reliability, and cost efficiency.\nDevelop reusable data engineering components, frameworks, and engineering patterns.\nImplement data quality, validation, reconciliation, and monitoring controls.\nTroubleshoot complex production issues, perform root cause analysis, and implement sustainable fixes.\nSupport CI/CD, testing, deployment, monitoring, and operational reliability.\nAct as a Data Engineering SME, providing technical guidance, design reviews, and code reviews.\nCollaborate with architecture, application, cloud, platform, and business teams to deliver enterprise data solutions.\nDrive automation, standardization, and continuous improvement across the data engineering landscape.\nRequired Skills\nStrong hands-on experience in Data Engineering.\nStrong hands-on experience with Databricks.\nAdvanced programming skills in Python and PySpark.\nStrong experience with AWS cloud-based data engineering.\nStrong proficiency in SQL.\nExperience designing and supporting enterprise ETL/ELT and data pipelines.\nStrong understanding of Apache Spark and distributed data processing.\nExperience with data lake and lakehouse architecture.\nStrong understanding of data modeling, data integration, and data quality principles.\nExperience with CI/CD and modern engineering practices.\nExperience supporting business-critical production data platforms.\nStrong troubleshooting, debugging, and root cause analysis skills.\nStrong communication and stakeholder-management capabilities.\nDesired Skills\nExperience with Snowflake data engineering and integration.\nKnowledge of Snowflake databases, schemas, warehouses, security, and performance optimization.\nExperience integrating Databricks/AWS data pipelines with Snowflake.\nExperience with GoldenGate, AWS DMS, or similar data replication technologies.\nKnowledge of data governance, lineage, metadata management, and auditing.\nExperience with data quality and reconciliation frameworks.\nExposure to enterprise monitoring and observability solutions.\nExperience working in financial services or other regulated environments.\nExperience Requirements\n10+ years of overall experience in software engineering, data engineering, or related technology roles.\n8+ years of hands-on data engineering & Devops experience.\n7+ years of relevant experience across Databricks, PySpark/Spark, and cloud-based data engineering preferred.\nProven experience designing, developing, and supporting enterprise-scale production data pipelines and platforms.\nDemonstrated ability to operate as a hands-on technical SME in complex enterprise environments.\nEducation Requirements\nBachelor’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related technical discipline.\nMaster’s degree in a relevant discipline is desirable.\nTechnology Stack\nCore / Mandatory:\nDatabricks | Python | PySpark | AWS | SQL | Apache Spark | ETL/ELT | Data Pipelines | Data Lake/Lakehouse\nDesired:\nSnowflake | GoldenGate | AWS DMS | Data Governance | Data Lineage | Data Quality | Observability\nAbout State Street\nAcross the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.\nWe are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.\nAs an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.\nDiscover more information on jobs at StateStreet.com/careers\nRead our CEO Statement","description_format":"text","description_chars":6022,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Flexible schedule","Hybrid work"],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Asset Management & Funds","Custody & Fund Administration"],"lifecycle":[{"event":"open","at":"2026-10-06T07:49:20Z"}],"visa":[],"liveness":{"score":65,"band":"ok","label":"Likely open","p_open":1,"p_active":0.868,"p_room":0.75,"age_days":15,"expected_fill_days":20,"reasons":["conf:2","velocity","win:late","comp:brand"],"computed_at":"2026-10-22T06:10:10Z"},"pay":null,"html_url":"https://alion.io/job/state-street-lead-data-engineer-assistant-vice-president","json_url":"https://alion.io/job/state-street-lead-data-engineer-assistant-vice-president.json","meta":{"generated_at":"2026-10-22T06:32:58Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1549,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1806},"rest":"https://alion.io/mcp/rest/get_company?id=1806"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fstate-street-lead-data-engineer-assistant-vice-president"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fstate-street-lead-data-engineer-assistant-vice-president"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fstate-street-lead-data-engineer-assistant-vice-president"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/state-street-lead-data-engineer-assistant-vice-president\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fstate-street-lead-data-engineer-assistant-vice-president"}]}