Senior Data Engineer

This role is for one of Weekday’s clients

Salary range: Rs 8000000 - Rs 20000000 (ie INR 80 - 200 LPA)

Min Experience: 5+ years

Location: Bengaluru

JobType: full-time

We are seeking an experienced Senior Data Engineer with 5-15 years of experience to design, build, and optimize scalable data systems and pipelines. The ideal candidate will have strong expertise in data mining, pattern identification, large-scale systems, data processing, and distributed architectures. You will work closely with engineering, analytics, product, and business teams to transform complex datasets into reliable, actionable data solutions.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable data pipelines, ETL/ELT workflows, and data processing systems.
• Perform advanced data mining across large and complex datasets to identify trends, relationships, anomalies, and valuable insights.
• Analyze datasets to identify meaningful patterns and correlations that can support business and product decisions.
• Build and optimize data architectures capable of handling large-scale data volumes and high-throughput workloads.
• Develop reliable batch and real-time data processing solutions using distributed computing technologies.
• Work with structured and unstructured data from multiple sources and ensure data is accurately processed and transformed.
• Optimize data pipelines, queries, storage, and processing frameworks for performance, scalability, and reliability.
• Implement data quality, validation, monitoring, and error-handling mechanisms across data workflows.
• Collaborate with Data Scientists, Analysts, Software Engineers, Product Managers, and other stakeholders to understand data requirements.
• Design and maintain data models, schemas, and data integration frameworks.
• Troubleshoot complex data and pipeline issues and drive improvements in system reliability and efficiency.
• Contribute to technical architecture decisions and establish best practices for large-scale data engineering.
• Mentor junior and mid-level engineers and contribute to improving engineering standards and processes.

Must-Have Skills

• Data Mining
• Strong understanding of pattern identification, trend analysis, and data exploration
• Experience working with Large Scale Systems
• Strong knowledge of data structures, algorithms, and database concepts
• Hands-on experience building ETL/ELT pipelines and data processing workflows
• Experience with SQL and relational databases
• Proficiency in at least one programming language such as Python, Java, or Scala
• Experience with distributed data processing and large-scale data platforms
• Strong understanding of data modeling, data warehousing, and data architecture
• Experience optimizing data pipelines and processing workloads for performance and scalability
• Strong analytical and problem-solving abilities

Good-to-Have Skills

• Experience with Apache Spark, Kafka, Hadoop, Flink, or similar distributed technologies
• Exposure to cloud platforms such as AWS, Microsoft Azure, or Google Cloud
• Experience with data lakes, lakehouses, and modern data warehouse technologies
• Knowledge of real-time and streaming data architectures
• Experience with data governance, observability, and data quality frameworks
• Familiarity with machine learning or analytics workflows is a plus
• Experience working with high-volume, high-velocity, or complex datasets