Software Integration Developer

Role Purpose :

The Software Integrations Developer will be the architect of connectivity within our digital ecosystem. You will design, develop, and maintain robust integration solutions that bridge the gap between disparate systems, ensuring seamless data flow and high-performance processing. By leveraging the Microsoft Azure stack, you will transform complex business requirements into scalable, automated integration workflows that drive organizational efficiency.

Key Accountabilities :

- Data Engineering & Pipeline Development : Design and implement end-to-end data pipelines (batch and streaming) using Azure Data Factory / Synapse Pipelines and Databricks.

- ETL/ELT Development : Develop scalable ETL/ELT workflows using PySpark, SQL, and Databricks notebooks to process large-scale datasets efficiently.

- Data Ingestion : Build and optimize data ingestion frameworks for structured and unstructured data from multiple enterprise and external sources.

- Databricks & Lakehouse Architecture : Lead the design and implementation of Databricks-based lakehouse architecture, including Delta Lake and medallion (bronze, silver, gold) data layers.

- Spark Optimization : Optimize Spark jobs for performance, scalability, and cost efficiency, including partitioning, caching, and cluster tuning.

- Data Governance & Processing : Implement data versioning, incremental processing, and ensure high data quality, consistency, and governance standards.

- Azure Data Platform : Utilize Azure services such as Data Factory, Data Lake Storage, Synapse Analytics, and Key Vault to build secure, scalable data solutions.

- Application & Systems Integration : Design, build, and maintain integration solutions using Azure Functions, Logic Apps, and Service Bus / Event Grid to connect line-of-business applications, SaaS platforms, and external services with the data platform.

- API Development & Management : Develop and expose secure, well-documented REST APIs and event-driven endpoints, handling publishing, versioning, throttling, and authentication through Azure API Management.

- Event-Driven & Real-Time Messaging : Implement reliable, decoupled event-driven and near-real-time integration patterns using Event Hub, Event Grid, and Service Bus for resilient message and data exchange across systems.

- Performance & Optimization : Monitor and continuously optimize data workflows for reliability, performance, and cost efficiency.

- DevOps & Governance : Manage code repositories, CI/CD pipelines, and deployments using Azure DevOps, ensuring high-quality documentation and best practices.

- Collaboration & Stakeholder Engagement : Work closely with business stakeholders and technical teams to translate requirements into scalable data solutions and provide technical leadership.

What Success Looks Like :

Reliable and scalable Azure integration architectures with 99.9% uptime, optimized data throughput, and minimal technical debt; clear technical and API documentation; strict adherence to security governance; and seamless data flow that earns positive feedback from business stakeholders.

Candidate Profile :

Experience & Capability :

- 5+ years of experience in Data Engineering / Data Platform development.

- 3+ years of hands-on experience with Azure Databricks.

- Strong experience with Azure Data Factory, ADLS Gen2, and Synapse Analytics.

- Proven experience in building enterprise-scale data pipelines.

- Hands-on experience building application and API integrations using Azure Functions, Logic Apps, and Azure API Management.

- Experience working in Agile/Scrum environments and full SDLC lifecycle.

Technical Strength :

- Development : Strong proficiency in Python (PySpark), SQL, and experience working with Databricks notebooks for scalable data processing and transformation logic.

- Data Design : Deep knowledge of ETL/ELT design patterns, including Slowly Changing Dimensions (SCD), medallion architecture (bronze, silver, gold), and data modelling for analytics.

- Platforms & Tools : Expert-level experience with Azure Data Factory / Synapse Pipelines, Azure Data Lake Storage, and Azure Databricks including Delta Lake.

- Integration & APIs : Hands-on experience designing integration and API solutions with Azure Functions, Logic Apps, Service Bus / Event Grid, and Azure API Management, including REST API design and secure authentication (OAuth 2.0 / managed identities).

- Databricks Expertise : Advanced hands-on experience with Spark optimization, cluster management, job orchestration, and performance tuning in Databricks.

- Bonus Strengths : Experience with streaming technologies, Microsoft Fabric, CI/CD for data platforms, or Power BI will be considered a significant advantage.

Mindset & Behaviours :

- Design-first and quality-focused mindset.

- Comfortable working in ambiguity and evolving requirements.

- Strong communicator able to explain technical concepts to non-technical audiences.

- Proactive, collaborative, and continuous improvement oriented.

- Demonstrates strong sense of urgency and ownership.

Skills

Solution Integration, Data Engineering, Data Pipeline, Azure Data Factory, ETL, PySpark, SQL, Data Governance, Azure Databricks, Python, API Integration