Data Engineer, Specialist

Provides expert level data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes the deepest dive analytical skillset on a variety of internal and external data. Leads, instructs, and mentors newer Data Engineering crew.

About Vanguard

Founded in 1975, Vanguard is one of the world's leading investment management companies. The firm offers investments, advice, and retirement services to tens of millions of individual investors around the globe—directly, through workplace plans, and through financial intermediaries.

Vanguard India

Vanguard's office in India is a significant milestone in our global expansion. We are committed to establishing an enduring technology center in Hyderabad, Telangana and are excited to be adding talent who will focus on Artificial Intelligence (AI), mobile, and cloud-based technologies that drive our business outcomes and deliver a world-class experience for our clients.

Role Summary

The Data Engineer, Senior Specialist is responsible for designing, developing and optimizing data pipelines and architectures to support scalable and efficient data processing. This role involves working with complex datasets, integrating multiple data sources and ensuring high data quality and reliability. The role requires a strong product mindset to understand business needs, translate them into sustainable technical solutions, and take end-to-end ownership of delivery, quality, reliability, and continuous improvement. The individual will collaborate with business and technology stakeholders to deliver secure, high-quality, and future-ready data solutions while contributing to modern AI-assisted engineering practices.

The role also carries technical ownership for business-critical data products and platforms, including production reliability, incident management, release support, operational improvements, and sustainable application-support practices.

Responsibilites

• Develop and maintain efficient ETL (Extract, Transform, Load) pipelines that ingest, process and transform large datasets from diverse sources. Build and manage data storage solutions using relational and non-relational databases, ensuring high availability, performance and scalability.
• Work with APIs, third-party data sources and internal databases to create unified datasets that provide actionable insights for business stakeholders.
• Partner with business intelligence analysts, data scientists and software engineers to design and implement data-driven solutions that meet business requirements.
• Design optimized data schemas and structures that support analytical workloads and improve query performance.
• Leverage orchestration tools like Apache Airflow or Prefect to automate ETL workflows and improve operational efficiency.
• Provide technical guidance, review code and promote best practices within the data engineering team.
• Lead and mentor Data and AI Engineers, providing technical direction, coaching, and engineering guidance.
• Lead solution architecture and technical design, ensuring alignment with business objectives, product priorities, and engineering standards.
• Design and optimize scalable data pipelines, cloud data platforms, data models, and storage solutions supporting analytical and operational workloads.
• Architect reliable batch and real-time data processing solutions, incorporating data quality, governance, security, performance, and scalability.
• Partner with Product Owners, business stakeholders, Architects, Analysts, and Engineering teams to translate requirements into sustainable data solutions.
• Take end-to-end technical ownership across design, development, deployment, production reliability, and continuous improvement.
• Evaluate technical options and make informed architecture decisions considering performance, scalability, security, cost, reliability, and maintainability.
• Establish engineering and operational practices covering CI/CD, automation, testing, observability, monitoring, release readiness, and production support.
• Lead complex production incident resolution, root-cause analysis, preventive actions, and reliability improvements, while guiding engineers in troubleshooting and reducing operational risk.
• Drive engineering excellence through architecture/design/code reviews, reusable patterns, DevOps practices, and AI-assisted development.
• Strong product mindset, connecting technical decisions and engineering investments to business and customer outcomes.
• Proven ability to lead, mentor, and influence engineers while remaining technically hands-on.
• Strong analytical, problem-solving, troubleshooting, and technical decision-making skills.
• Effective communication and stakeholder management across technical, business, and product teams.
• Ability to navigate ambiguity, balance competing priorities, and proactively manage risks, dependencies, and trade-offs.
• Strong commitment to engineering excellence, operational reliability, and continuous improvement.
• Willingness to provide technical leadership during critical production incidents, major releases, and business-critical situations outside regular working hours when required.
  
  

Qualifications and Skills

• Minimum 8 years of relevant work experience, with at least 4 years focused on designing and developing scalable data pipelines, ETL frameworks and database management systems.
• Bachelor's degree (B.E./B.Tech) in Computer Science or IT, or Bachelor's in Computer Applications (BCA) from a recognized institution.
• Proficiency in SQL and database technologies such as PostgreSQL, MySQL, Oracle, or SQL Server.
• Hands-on experience with big data frameworks like Apache Spark, Hadoop, or Databricks.
• Strong programming skills in Python, Scala, or Java for data processing and automation.
• Experience with cloud platforms such as AWS, Azure, or GCP, particularly in data services like AWS Glue, Redshift, Big Query, or Snowflake.
• Deep hands-on experience architecting and delivering AWS-based data solutions using services such as S3, Glue, Lambda, Redshift, Athena, Step Functions, SNS/SQS, IAM, CloudWatch, Kinesis, and EMR. Ability to make architecture decisions covering scale, security, reliability, performance, and cost.
• Hands-on experience or strong working knowledge of Amazon Bedrock, including Knowledge Bases, Guardrails and Agents, along with SageMaker, Lambda, S3, OpenSearch Serverless vector capabilities, Step Functions, and infrastructure as code using CDK or CloudFormation.
  
  

Location


This role is based in Hyderabad, Telangana at Vanguard India. Only qualified external applicants will be considered.

Our mission


Vanguard adheres to a simple purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investment success.

Our commitment to you

Vanguard takes the same long-term view of your success—at work and in life—with Benefits and Rewards packages that reflect what you care about, throughout all the phases and stages of your life. Our Total Rewards programs provide you and your loved ones with wellness support for key areas in your life:

Financial wellness

We're committed to enabling your financial success and provide competitive offers and programs.

Physical wellness

We're committed to providing benefits that support your physical health and wellness.

Personal wellness

We're committed to providing resources that help support the full scope of your life.

How We Work

Vanguard has implemented a hybrid working model for most of our employees (crew members), designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.