Azure Data Engineer with Databricks

Azure Data Engineer with Databricks

THE WORK:

• Develop and implement scalable data processing solutions on Azure, transforming and cleaning large structured and unstructured datasets efficiently.
• Design, build, and maintain robust data pipelines integrating data from diverse sources (databases, APIs, streaming platforms).
• Collaborate with cross-functional teams including data engineers, data scientists, and cloud architects to deliver end-to-end data platform solutions.
• Engage at all critical stages of project development with business analysts, source system experts, data architects, engineers, and product owners.
• Clearly communicate and document technical work, factoring in performance and cost considerations in all design decisions.

HERE’S WHAT YOU’LL NEED:

• Hands-on experience with implementing solutions with data lakes and pipelines (Databricks, Azure Data Factory).
• Proven experience in designing and implementing solutions in Azure cloud (knowledge of AWS, GCP clouds will be an additional advantage).
• Proficiency in SQL, PySpark, data modeling (3NF, star/snowflake schemas) and ELT/ETL design patterns (knowledge of popular ETL tools, including Azure Data Factory).
• Experience with Python or similar scripting languages for data transformation, orchestration, and automation.
• Familiarity with advanced Databricks features (Unity Catalog, MLflow, Jobs & Pipelines).
• Understanding cloud data architecture principles, including data Lakehouse concepts, Medallion architecture, API integration, and event-driven architecture.
• Experience with CI/CD and DevOps practices for data workflows (Git, Azure DevOps) and analytics integration (e.g., Power BI).
• Excellent communication and client-facing skills, with the ability toexplain technical concepts clearly to team members and stakeholders.
• Strong collaboration skills across multidisciplinary teams.

Data Architect with Azure and Databricks

THE WORK:

• Design and implement scalable cloud-based data warehousing and analytics solutions using Azure Data Lake, Azure Databricks, Microsoft Fabric and related services (e.g., Power BI).
• Defining best practices for data governance, security, performance optimization, and cost efficiency in Azure and Databricks environments.
• Collaborating with cross-functional teams (data engineers, data scientists, and cloud architects) to deliver end-to-end data platform solutions.
• Integrate Databricks with the broader Azure ecosystem, including data lakes, AI/ML frameworks, reporting systems, and data catalogs (e.g., Unity Catalog, Purview).
• Advising clients on modernization strategies, including migration from legacy on-premises databases to Azure and Databricks-based architectures.
• Supporting pre-sales and solutioning activities, including estimation, proof-of-concepts, and technology assessments.
• Staying up to date with innovations in Azure, Databricks, Fabric and applying industry trends to ongoing projects.
• Collaborating across all crucial stages of project development with business analysts, source system experts, data architects, engineers, and product owners.
• Proposing and defending effective, consistent solutions that balance strategic development and existing architecture.
• Communicating and documenting work clearly, considering performance and cost factors in design decisions

HERE’S WHAT YOU’LL NEED:

• Strong hands-on experience designing and implementing data platforms in Azure and Databricks, including performance tuning, security setup, automation, and data sharing.
• Proven experience in designing and implementing data platforms in Azure cloud (knowledge of AWS, GCP clouds will be an additional advantage).
• Expertise in SQL, PySpark, data modeling (3NF, star/snowflake schemas) and ELT/ETL design patterns (knowledge of popular ETL tools, including Azure Data Factory).
• Proficiency in Python or similar scripting language s for data transformation, orchestration, and automation.
• Experience with advanced Databricks features (Unity Catalog, MLflow, Jobs & Pipelines and others).
• Deep understanding of cloud data architecture principles, including data Lakehouse concepts, Medallion architecture, API integration, and event-driven architecture.
• Possesses comprehensive knowledge of Azure Databricks and Microsoft Fabric architecture, encompassing familiarity with their core components, integration strategies, and established best practices for developing scalable, secure, and efficient data solutions.
• Experience with CI/CD and DevOps practices for data workflows (Git, Terraform, Azure DevOps, etc.).
• Excellent communication and client-facing skills with the ability to translate business needs into technical solutions.
• Proven ability to work collaboratively across multidisciplinary teams.