AWS, Python - Sr Software Engineer II

Job Position:

We are seeking a highly skilled and motivated Cloud Data Engineering to join our team. The ideal candidate will have extensive experience in building and managing cloud-based data platforms, with expertise in SQL, AWS, Python, and Snowflake. This role requires a strong background in data engineering, cloud ingestion, ETL, and cloud migration to design, implement, and lead scalable and efficient data solutions.

Job Responsibility:

• Lead the design and implementation of cloud ingestion and migration platforms.
• Develop robust and scalable ETL pipelines for data processing and transformation.
• Leverage AWS and GCP to design, deploy, and maintain scalable cloud-based architectures.
• Optimize data storage, retrieval, and analytics solutions using Snowflake, BigQuery, and Databricks.
• Implement solutions for real-time and batch data ingestion.
• Ensure data pipelines are reliable, scalable, and secure.
• Work closely with cross-functional teams to gather requirements and translate them into technical designs.
• Communicate progress and technical challenges effectively to stakeholders.

Qualification:

• Minimum education: B.Tech degree/MCA with computer science background or equivalent experience.
• Preferred education: BE/B.Tech/MCA.

Experience:

• 5+ years of hands-on experience in data engineering.
• Proficiency in AWS and Python for building data pipelines.
• Strong experience in building ETL pipelines and workflows.

Skills and Competencies:

Technical Expertise:

• 5+ years of hands-on experience in data engineering.
• Proficiency in AWS and Python for building data pipelines.
• Deep knowledge of cloud platforms, particularly AWS and GCP.
• Expertise in Snowflake for data storage, querying, and analytics.
• Knowledge of Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, Workflows, and Performance Optimization.
• Strong proficiency in Python, PySpark, Spark SQL, and distributed data processing.

Cloud Migration & Architecture:

• Experience in designing and executing large-scale cloud migration projects.
• Ability to architect secure and cost-optimized cloud solutions.
• Hands-on experience in industry standard cloud migration framework and architecture is must.
• Architecture of implementing PHI/PII tokenization processes.

Soft Skills:

• Strong problem-solving and analytical skills.
• Excellent written and verbal communication.
• Ability to lead and work collaboratively in a team environment.

Skills

AWS, BigQuery, Databricks, GCP, PySpark, Python, Snowflake, Spark SQL