Big Data Developer

Responsibilities:

• Develop and Design: Lead the design and development of specifications, develop and build code, perform code reviews, test, debug, document, and manage changes to ensure the successful implementation and maintenance of programs.
• Cloud Experience: Utilise extensive experience with Microsoft Azure, including Azure Data Factory, Databricks, Azure Data Lake Storage, Azure Blob, Azure DevOps, Azure Monitor, Azure Synapse, Azure Data Catalogue/Purview, and Cosmos DB to deliver robust data solutions.
• Data Management: Apply proficiency in Microsoft Purview for data quality and catalogue management, and demonstrate strong knowledge of data quality and data governance practices.
• Analysing Business Requirements: Research and analyse business requests, recommend solutions, and prepare design specifications in collaboration with senior team members.
• Coordination: Participate in production and user acceptance testing support to troubleshoot complex problems on applications and technologies. Coordinate change ticket implementations and effectively interact with stakeholders, end users, business analysts, and technical resources.

Technical Tools:

Leverage development and automation tools including Maven, Jenkins, GitHub, and Bash scripting to enhance productivity and quality of deliverables.

Cross-Functional Relationships:

• Collaborates with peers within the Big Data Platform Services team and other technology groups to identify, discuss, and resolve business issues related to data management and application development.
• Interacts regularly with business analysts, end users, and stakeholders to gather requirements and ensure alignment of solutions with business objectives.

Job Dimensions:

• Scope includes supporting enterprise-wide data initiatives, covering multiple geographies and business functions.
• Responsible for the design, development, and maintenance of data solutions impacting various lines of business.
• Provides advisory and technical expertise to ensure successful implementation and adoption of big data technologies.

Requirements:

• 5+ years of experience as an Application Developer.
• Strong understanding of data quality, data governance, and catalogue management principles.
• Proficient in Agile/Scrum methodologies and application/architecture development.
• Azure Data Cloud Services (including Azure Data Factory, Databricks, Azure Data Lake Storage).
• Skills: SQL, Python, Java or Scala, Spark, Big Data- Hadoop (Hive or Sqoop).
• Strong analytical, problem-solving, and communication skills.
• Ability to work collaboratively within a team and influence outcomes through effective relationship building.

Good to have Skills:

• Azure DevOps, Azure Monitor, Azure Synapse, Azure Data Catalogue/Purview.
• Cosmos DB.
• Microsoft Purview.
• Maven.
• Jenkins.
• GitHub.

Skills

ADF, Azure, Databricks, Hadoop, Hive, Java, Python, SQL, Scala, Spark, Sqoop