Software Engineer - Data Engineering

We are seeking a highly skilled and experienced Senior Software Engineer to join our innovative team at McKesson Compile India Private Limited. In this role, you will be instrumental in designing, developing, and maintaining complex software solutions that drive our business forward. You will contribute to the full software development lifecycle, ensuring high-quality, scalable, and secure applications.

Responsibilities:

• Design and implement scalable and maintainable data pipelines aligned with business requirements
• Design and build web crawlers using Scrapy and Python.
• Independently own and deliver medium to large features, enhancements, and technical initiatives.
• Participate in technical design discussions, evaluate implementation alternatives, and estimate development effort.
• Define and implement automated testing, data quality checks, and monitoring capabilities.
• Troubleshoot production issues, perform root cause analysis, and implement preventive solutions.
• Optimise application, database, and pipeline performance, scalability, reliability, and cost efficiency.
• Contribute to CI/CD, automation, observability, and operational excellence initiatives.
• Review peer code and promote engineering best practices across the team.

Requirements:

• Degree or equivalent; typically requires 2+ years of relevant experience.
• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
• 2+ years of professional experience in software development, with a proven track record of delivering high-quality software solutions.
• Strong proficiency in at least one modern programming language such as Python.
• Strong knowledge of data structures, object-oriented programming principles and design patterns.
• Proficient in building scalable data pipelines using Apache Spark or PySpark.
• Proficient in writing complex SQL queries, CTE's, joins and window functions for data transformations
• Experience in orchestrating complex workflows using tools like Apache Airflow or Prefect.
• Experience with relational and/or NoSQL databases (e. g., PostgreSQL, Snowflake).
• Experience with cloud platforms (e. g., AWS, Azure, GCP), data platforms and distributed systems.
• Proficient with GitLab/GitHub, Git CLI and working with team workflows like Git-flow is a plus.
• Experience in implementing web crawlers using Scrapy and Python is a plus.

Preferred Skills:

• Experience implementing CI/CD for data pipelines with automated testing and data quality checks is a plus.
• Familiarity with microservices architecture, RESTful APIs, and distributed systems.
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and collaboratively in a fast-paced, dynamic environment.
• Experience with healthcare domain knowledge is a plus.