Data Engineer

About Position:

We are seeking a highly skilled Senior Data Engineer to contribute to the data management and engineering initiatives supporting Test & Learn and other analytics-driven products. The ideal candidate will have extensive experience in building scalable data pipelines, optimizing data processing frameworks, and delivering high-quality data solutions in cloud environments. You will work closely with data engineers, analysts, and business stakeholders to design and implement enterprise-grade data platforms that enable advanced analytics and business experimentation capabilities.

• Role: Senior Data Engineer - PySpark & Databricks
• Location: All Persistent Locations
• Experience: 8 to 12 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable enterprise data pipelines and ETL/ELT solutions.
• Support the implementation and maintenance of cloud-based data platforms for a global client base.
• Build robust and highly available data engineering solutions using PySpark, Python, SQL, and Databricks.
• Develop efficient data processing frameworks capable of handling large-scale datasets.
• Optimize SQL queries and data workloads to improve performance and scalability.
• Partner with analytics teams and business stakeholders to understand business requirements and translate them into technical data solutions.
• Design and implement data transformation, cleansing, and validation processes to ensure data quality and accuracy.
• Collaborate with Data Engineers, Analysts, and Architects to deliver accurate, reliable, and business-ready datasets.
• Participate in code reviews and ensure adherence to software engineering and data engineering best practices.
• Implement automated data validation, monitoring, and quality control mechanisms.
• Support data platform modernization and continuous improvement initiatives.
• Contribute to DevOps and CI/CD processes for data engineering workflows.
• Build strong relationships with internal teams and clients across global delivery environments.
• Work on multiple projects simultaneously while ensuring timely and high-quality delivery.
• Ensure compliance with organizational data governance, security, and regulatory requirements.

Expertise You'll Bring:

• 8 to 12 years of experience in Data Engineering, Big Data Engineering, or Data Platform Development.
• Strong hands-on experience with PySpark for large-scale data processing and transformation.
• Advanced proficiency in SQL with strong query optimization and performance tuning expertise.
• Extensive experience using Python for data engineering, automation, and data transformation tasks.
• Hands-on experience working with Databricks and cloud-based data platforms.
• Strong understanding of ETL/ELT frameworks and enterprise data integration practices.
• Experience designing and maintaining scalable data pipelines supporting analytics and reporting workloads.
• Knowledge of modern data warehousing concepts and dimensional data modeling techniques.
• Experience processing large datasets in distributed environments.
• Familiarity with Azure cloud services and cloud-native data engineering solutions.
• Understanding of DevOps principles, version control systems, and CI/CD practices.
• Ability to perform code reviews with a focus on performance, maintainability, testability, and code quality.
• Strong analytical, troubleshooting, and problem-solving skills.
• Experience working in Agile development environments and cross-functional teams.
• Excellent communication and stakeholder management skills.
• Ability to manage multiple priorities and deliver solutions in a fast-paced environment.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.