Senior Big Data Associate

As the Senior Associate, People, and Senior Associate L2 in Data Engineering, you will translate client requirements into technical design and implement components for data engineering solutions. Utilize a deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to ensure the necessary health of the overall solution.

Responsibilities:

• Data Ingestion, Integration, and Transformation.
• Data Storage and Computation Frameworks, Performance Optimizations.
• Analytics and Visualizations.
• Infrastructure and Cloud Computing.
• Data Management Platforms.
• Build functionality for data ingestion from multiple heterogeneous sources in batch and real-time.
• Build functionality for data analytics, search, and aggregation.

Requirements:

• Minimum 5 years of experience in Big Data technologies.
• Hands-on experience with the Hadoop stack - HDFS, sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, Hive, oozie, Airflow, and other components required in building end-to-end data pipelines. Working knowledge of real-time data pipelines is an added advantage.
• Strong experience in at least the programming languages Java, Scala, and Python. Java is preferable.
• Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDB, Cassandra, AWS Redshift, Azure SQL DW, GCP BigQuery, etc.
• Well-versed and working knowledge of data platform-related services on Azure.
• Bachelor's degree and year of work experience of 6 to 8 years or any combination of education, training, and/or experience that demonstrates the ability to perform the duties of the position.
• Good knowledge of traditional ETL tools (Informatica, Talend, etc. ) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands-on experience.
• Knowledge of data governance processes (security, lineage, catalog) and tools like Collibra, Alation, etc.
• Knowledge of distributed messaging frameworks like ActiveMQ/RabbitMQ/Solace, search and indexing, and Microservices architectures.
• Performance tuning and optimization of data pipelines.
• Cloud data specialty and other related Big Data technology certifications.