Lead Engineer

Role Overview :

Experience : 4-8 yrs.

Age Limit : Maximum 35 Years.

Location : On-Site / Hybrid.

Employment Type : Full-Time.

We are seeking a high-performing Lead Engineer (API & Data Engineering) to drive our integration platforms and Lakehouse ecosystem. This role sits at the intersection of modern cloud-native API development and large-scale data engineering.

You will be responsible for designing and deploying high-performance REST APIs (FastAPI) alongside robust, scalable ETL/ELT data pipelines utilizing Databricks, PySpark, Azure Data Factory (ADF), and AKS/ACA. If you excel at bridging technical architecture with stakeholder management across cross-functional business lines, this role offers direct impact and visibility.

Key Competencies :

- Hands-on Problem Solver : Proven ability to troubleshoot, debug, and optimize complex data pipelines and cloud infrastructure.

- Production Governance : Deep understanding of secure data exchange, authentication protocols, and compliance across multi-tenant systems.

- Agile Leadership : Ability to manage day-to-day delivery autonomously while consulting leadership on high-impact architectural decisions.

Key Responsibilities :

API & Microservices Engineering :

- Design, build, and maintain scalable, low-latency REST APIs using FastAPI and Python.

- Implement microservices architecture, secure API endpoints, and optimize runtime performance under high-concurrency workloads.

Data Engineering & Lakehouse Management :

- Build and optimize enterprise data pipelines using Databricks (PySpark, SQL) and Azure Data Factory (ADF).

- Perform advanced data modeling, schema design, and query optimization for high-volume analytics.

Cloud-Native Deployments & Containerization :

- Package microservices and pipelines using Docker.

- Deploy, manage, and monitor containerized workloads on Azure Kubernetes Service (AKS) and Azure Container Apps (ACA).

Stakeholder Management & Technical Governance :

- Independently drive maintenance, bug fixes, operational readiness, and stakeholder interactions.

- Collaborate with internal business verticals (Wheels, Collections, Emerging Business) and external partners (FSS Subsidiaries, Data Vendors) to translate requirements into technical solutions.

- Partner with Engineering Management on major architectural calls, infrastructure investments, and technology roadmaps.

Duties & Responsibilities :

- Design, develop, and maintain API-led integration solutions and data engineering pipelines to support business applications.

- Build and enhance scalable REST APIs using frameworks such as FastAPI, ensuring high performance and reliability.

- Develop and optimize data processing workflows using Databricks (PySpark, SQL) for efficient data transformation and analytics.

- Collaborate with business and technical teams to understand requirements and translate them into robust API and data solutions.

- Implement containerized applications using Docker and deploy/manage services on cloud platforms such as Azure Container Apps (ACA) and AKS.

- Develop and manage data integration workflows using Azure Data Factory (ADF) or equivalent to enable seamless data movement across systems.

- Ensure adherence to best practices in API design, microservices architecture, security, and performance optimization.

- Troubleshoot, debug, and resolve issues related to APIs, data pipelines, and cloud deployments.

- Stay updated with emerging technologies and continuously improve technical skills in API engineering, cloud-native development, and data platforms.

- External Stakeholders : FSS Subsidiary members and Data vendors.

5. Key Challenges :

- Ensuring high performance, scalability, and reliability of REST APIs under varying workload conditions.

- Managing secure API integrations and data exchange across multiple systems with proper authentication and governance.

- Deploying, monitoring, and optimizing containerized applications on Docker, ACA, and AKS environments.

- Handling large-scale data processing and optimization in Databricks (PySpark, SQL) while maintaining efficiency.

- Troubleshooting and maintaining stability across APIs, data pipelines, and cloud-native infrastructure.

Skills :

- Develop scalable and high performance REST APIs using FastAPI.

- Strong hands-on experience with Databricks (PySpark, SQL) for data processing.

- Experience in building and managing data pipelines and integrations.

- Proficiency in SQL for data querying, transformation, and optimization.

- Hands-on experience with Docker and deployment on ACA/AKS environments.

- Ability to develop and manage workflows using Azure Data Factory (ADF) or equivalent.

- Good understanding of database design and data architecture.

- Strong debugging, troubleshooting, and performance tuning skills.

- Python programming for API development and data engineering use cases.

Skills

Data Engineering, Azure Databricks, PySpark, ETL, Native Cloud, DataLake, SQL, Data Pipeline, Data Integration