Senior Software Engineer Data Platform Engineering

Senior Software Engineer – Data Engineering (3-5 Years Experience)

Myntra Data Platform (MDP) About The Role

• Myntra Data Platform (MDP) powers the data foundation behind analytics, experimentation, personalization, machine learning, and critical business decisions across Myntra.
• We are looking for a Senior Software Engineer – Data Engineering who is passionate about building scalable data platforms and production-grade data pipelines. You will work on large-scale batch and near real-time data processing systems, drive data quality and reliability, and contribute to the evolution of Myntra's modern lakehouse architecture.This role requires strong software engineering fundamentals combined with hands-on expertise in Apache Spark, SQL, Python, ETL development, workflow orchestration, data modeling, and modern data platforms.You will collaborate with platform, analytics, product, and engineering teams to build highly reliable and scalable data solutions that power business-critical use cases across the organization.

What You'll Do

Data Pipeline Engineering

• Design, develop, and maintain scalable ETL/ELT pipelines for large-scale data processing.
• Build reliable data ingestion, transformation, validation, and publishing workflows.
• Develop reusable components and frameworks to improve engineering productivity.
• Implement incremental processing, backfills, retries, and failure recovery mechanisms.
• Ensure data pipelines meet reliability, scalability, and performance requirements.
• Apache Spark & Big Data Processing
• Build and optimize large-scale data processing jobs using Apache Spark, PySpark, and Spark SQL.
• Develop production-grade transformations, aggregations, and data enrichment pipelines.
• Analyze Spark execution plans and optimize jobs for performance and cost.
• Troubleshoot issues related to partitioning, joins, data skew, shuffles, and memory utilization.
• Follow engineering best practices for efficient distributed processing.
• Data Modeling & Warehousing
• Design and implement scalable data models for analytics and reporting.
• Build curated datasets supporting business intelligence and downstream applications.
• Apply dimensional modeling concepts including facts, dimensions, star schemas, and SCDs.
• Translate business requirements into reusable and maintainable data models.
• Workflow Orchestration
• Develop and manage workflows using Apache Airflow or equivalent orchestration platforms.
• Define task dependencies, scheduling strategies, retries, and monitoring mechanisms.
• Improve operational visibility and reliability of data workflows.
• Support self-service capabilities for pipeline onboarding and deployment.
• Lakehouse & Data Platform Engineering
• Build and operate workloads on Databricks and Spark-based platforms.
• Work with Delta Lake, Apache Iceberg, Parquet, or similar storage formats.
• Optimize storage layouts, partitioning strategies, compaction, and metadata management.
• Contribute to platform capabilities that simplify data consumption and processing.
• Data Quality & Governance
• Implement automated data quality validations and monitoring frameworks.
• Develop checks for freshness, completeness, schema validation, and consistency.
• Ensure proper metadata management, lineage tracking, and discoverability.
• Follow data governance and security best practices.
• Operational Excellence
• Monitor pipeline health, SLAs, performance, and cost metrics.
• Investigate production issues and perform root cause analysis.
• Improve platform stability through automation and continuous optimization.
• Participate in on-call support and incident management when required.
• Engineering Excellence
• Write clean, maintainable, and testable code.
• Participate in code reviews and design discussions.
• Contribute to engineering standards, best practices, and reusable frameworks.
• Collaborate with cross-functional teams to deliver high-quality solutions.

What We're Looking For

Required Qualifications

• 3-5 years of hands-on experience in Data Engineering or Software Engineering.
• Strong experience with Apache Spark, PySpark, and Spark SQL.
• Proficiency in Python and strong coding fundamentals.
• Excellent knowledge of SQL and query optimization techniques.
• Experience building production-grade ETL/ELT pipelines.
• Hands-on experience with Apache Airflow or similar workflow orchestration tools.
• Good understanding of data warehousing and data modeling concepts.
• Experience working with Databricks, Hadoop, or cloud-based data platforms.
• Familiarity with Delta Lake, Apache Iceberg, or Parquet.
• Strong debugging, troubleshooting, and problem-solving skills.
• Good understanding of software engineering best practices, CI/CD, and version control systems.

Preferred Qualifications

• Experience working with Kafka or other streaming technologies.
• Exposure to near real-time data processing architectures.
• Experience with dbt or similar SQL transformation frameworks.
• Knowledge of modern lakehouse architectures and cloud-native data platforms.
• Understanding of data governance, metadata management, and lineage concepts.
• Exposure to AI-assisted development tools and data engineering automation.
• Experience with Kubernetes, Docker, or cloud platforms such as GCP, AWS, or Azure