Senior Staff and Staff Software Engineer

Responsibilities:

• Architect, design, and deliver large-scale cloud-native data platforms primarily on AWS, with exposure to Azure and GCP, using Databricks and distributed processing frameworks to build highly scalable and resilient systems.
• Work hands-on across the technology stack - including Python, PySpark, Apache Spark, Databricks, AWS services, event-driven architectures, and SQL/NoSQL databases - to solve complex engineering challenges and maintain platform excellence.
• Lead organisation-wide technical initiatives focused on performance optimisation, scalability, reliability, security, governance, and cost efficiency.
• Review and influence critical architectural decisions across teams while establishing engineering best practices and architectural governance standards.
• Partner closely with global engineering, product management, architecture, and business stakeholders to align technical solutions with strategic business objectives.
• Own the end-to-end software development lifecycle, including requirements gathering, solution design, development, deployment, observability, and documentation.
• Mentor and guide senior and junior engineers, fostering a culture of innovation, accountability, collaboration, and technical excellence.

Requirements:

• B. E/B. Tech/M. Tech/MCA in Computer Science, Information Technology or a related field.
• 10-13 years of strong software engineering experience, with significant expertise in large-scale data engineering and distributed systems architecture.
• Solid expertise in Data Warehousing, Data Lakes, Delta Lake architecture, and modern big data ecosystem designs.
• Deep hands-on expertise in Databricks, Python, PySpark and Apache Spark, with proven experience building high-performance distributed data processing solutions and managing massive-scale datasets for analytics and business intelligence workloads.
• Strong experience with AWS services such as S3 Glue, Redshift, EMR, Athena, Lambda, and EventBridge for building scalable and reliable cloud-native data platforms.
• Experience with real-time and near real-time streaming technologies including Kafka, AWS Kinesis, SQS, and RabbitMQ.
• Strong understanding of both relational and NoSQL databases, including PostgreSQL, SQL Server, Aurora, RDS, MongoDB, and DynamoDB.
• Hands-on experience with Infrastructure as Code (IaC) tools such as Terraform or Ansible.
• Strong understanding of CI/CD and DevOps practices using tools such as Jenkins, Github/GitLab, Bitbucket, GoCD and automated deployment pipelines.
• Proven ability to influence technical direction, drive multi-functional initiatives, and lead complex engineering programs at scale.
• Exposure to Generative AI technologies including LLMs, RAG architectures, and Agentic AI systems, with experience designing and deploying AI-driven solutions.

Nice to have:

• AWS and Databricks certifications and experience with Azure / Google Cloud Platform (GCP)
• Working knowledge of TypeScript, Node. JS, and UX design and development.
• Experience building data platforms in privacy-safe or cleanroom environments.

Skills

amazon web services, aws, databricks, python