Big Data Engineer

Role: Engineer. Skills: Big Data, Spark, Hive. Experience: 5 - 18 Years

Inclusion without Exception

Tata Consultancy Services (TCS) is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity is reflected in our people stories across our workforce and implemented through equitable workplace policies and processes.

TCS has been serving the Canadian marketplace for more than 35 years and today is the technology partner of choice for many of the country's leading organizations. As one of the largest IT services providers in Canada and globally, TCS is aiming to become the world's largest AI-led technology services company and is enabling its clients to transform themselves across the full AI stack, from infrastructure to intelligence.

Rooted in the heritage and values of the Tata Group, TCS is focused on creating long-term value for its clients, investors, employees, and the communities it serves. With a highly skilled workforce supporting customers from coast to coast, TCS has consistently been recognized as a Top Employer by the Top Employers Institute, including a recent top ranking at the country level.

TCS also sponsors 14 of the world's most prestigious marathons and endurance events, including the TCS Toronto Waterfront Marathon, reflecting its commitment to health, sustainability, and community

Role Description

• Design and develop Big Data pipelines using Spark and Scala to process large volumes of raw data stored in Hive/Hadoop.
• Analyze raw Hive tables, source schemas, data dictionaries and source-to-target mappings to understand business data.
• Develop Spark/Scala transformation frameworks to convert raw source data into standardized conformed data views/tables.
• Implement complex business transformation rules including joins, filters, aggregations, derivations and reference-data lookups.
• Standardize data across multiple source systems to create a consistent enterprise/conformed data model.
• Implement incremental processing, partition-based processing and historical data handling for high-volume datasets.
• Develop reusable Spark/Scala components for data enrichment, transformation and validation.
• Implement data-quality controls for completeness, uniqueness, validity, referential integrity and business-rule validation.

Salary Range - CA$ 100,000 - CA$ 130,000 Per Year

Note:

TCS does not use artificial intelligence tools for candidate screening or evaluation.

This posting is for a current vacancy

The hiring process includes an initial screening by the TCS Hiring Team, followed by a technical evaluation and managerial discussion conducted by the Business Team, and concluding with the final HR evaluation.

Tata Consultancy Services Canada Inc. is committed to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodation during the recruitment and selection process, please inform Human Resources.

Thank you for your interest in TCS. Candidates that meet the qualifications for this position will be contacted within a 2-week period. We invite you to continue to apply for other opportunities that match your profile.