L2/L3 Application Support Engineer - Azure Data Factory & Databricks

At Fujitsu, our purpose is to make the world more sustainable by building trust in society through innovation. Founded in Japan in 1935, Fujitsu has been a pioneer in technology and innovation for decades. Today, as a world-leading digital transformation partner, we are committed to transforming business and society in the digital age.

With approximately 130,000 employees across over 50 countries, Fujitsu offers a broad range of products, services, and solutions. We collaborate with our customers to co-create solutions that drive enterprise-wide digitalization while actively working to address social issues and contribute to the United Nations Sustainable Development Goals (SDGs).

1. Position purpose

Provide second- and third-level support for Cleanaway's Azure batch data integration platform. This covers Azure Data Factory pipelines, Azure Databricks notebooks, self-hosted integration runtimes, pipeline Function Apps and supporting Azure services. Together these move HR, identity, procurement, finance, fuel and operational data between enterprise systems. The engineer resolves pipeline incidents, monitors scheduled processing, administers the platform and delivers enhancements, including completing remaining SSIS-to-ADF migrations through change control.

2. Key accountabilities

Incident management and service restoration

• Own assigned ServiceNow incidents through resolution: pipeline and notebook failures, missing or duplicate data, authentication and API failures, and integration-runtime outages.
• Diagnose failures using ADF monitoring, Databricks job runs, Spark logs and source and target data. Rerun, backfill or deduplicate under documented procedures.
• Prioritise feeds with downstream business impact, such as identity lifecycle (joiner/leaver), procurement and accounts payable, finance claims and executive reporting.

Monitoring and operational health

• Monitor scheduled ADF triggers, Databricks jobs and integration runtime health daily. Action alerts and failures before business deadlines.
• Disable and re-enable dependent pipelines around planned source-system outages (e.g. ERP maintenance windows) and validate catch-up processing.
• Build and tune Azure Monitor alerts so that failures are detected proactively rather than reported by users.

Integration and data support

• Investigate data issues across connected systems such as JDE, Coupa, Fieldglass, Dayforce, Preceda, PageUp, identity governance (SailPoint), fuel card providers and the data warehouse.
• Validate source and target data before attributing faults. Coordinate with SaaS vendors and internal application teams on API and file contract changes.
• Fulfil authorised data adjustment and file reprocessing requests with audit evidence.

Platform administration

• Administer linked services, Key Vault secrets, managed identities and service principals, integration runtime servers and DSNs within delegated Azure RBAC.
• Maintain Azure DevOps repositories and CI/CD pipelines (ARM, DACPAC), environment promotion and branching for the platform.
• Maintain the credential, secret and certificate expiry register, and rotate credentials before expiry.

Change, release and enhancement

• Design, build, test and deploy enhancements to pipelines, notebooks (PySpark/SQL) and Function Apps through Azure DevOps and Cleanaway change control.
• Complete remaining SSIS-to-ADF migrations as planned enhancements, working with the data warehouse engineer: parallel running, reconciliation and decommissioning.
• Take delivered project integrations into BAU through hypercare and documented handover, and manage the integration backlog with owner and status for each item.

Problem management and knowledge

• Raise problem records for recurring pipeline failures and dual-running risks, and drive permanent fixes.
• Maintain interface definition documents, runbooks, pipeline inventories and knowledge articles in the agreed repository.
• Participate in knowledge transfer from the incumbent provider, demonstrate independent handling of common support scenarios, and transfer knowledge to new team members.

Security and compliance

• Use privileged and production access only through the approved access process (e.g. CyberArk), on a least-privilege basis and for authorised tasks.
• Never record passwords, tokens or other secrets in tickets, email or documents. Store and hand over credentials only through the approved vault.
• Handle customer, employee and payroll personal information in line with the Privacy Act and Cleanaway data classification, including compliant incident records and escalation of potential data breaches.
• Maintain segregation of duties: perform data corrections, approvals and production changes only with documented business or change approval, and keep an audit trail.

3. Role boundaries

L2/L3 accountability

• Own monitoring, incident resolution, platform administration and enhancement delivery for ADF, Databricks and associated batch integration components.
• Own Function Apps and Logic Apps that run as steps within batch pipelines. Event-driven and API-based Logic Apps and Function Apps belong to the Azure Integration Services engineer.
• Retain ServiceNow ownership while vendors or application teams remediate source or target system faults.

Outside delegated authority

• Azure landing-zone, network and subscription architecture, and platform design decisions owned by Cloud and Architecture teams.
• Business decisions on data ownership, mapping rules and corrections in source systems.
• Leadership of new integration programmes. The engineer contributes delivery effort under change control.

4. Skills and experience

Essential

• Demonstrated enterprise production support and development experience with Azure Data Factory and Azure Databricks (PySpark, SQL, Delta Lake).
• Self-hosted integration runtimes, linked services, Key Vault, managed identities and Azure RBAC.
• Azure DevOps CI/CD for data platforms (ARM templates, DACPAC, YAML pipelines).
• Strong SQL skills and experience with REST API and file-based integrations with SaaS platforms.
• ITIL-aligned incident, problem, change and request practice using ServiceNow or equivalent.

Desirable

• SSIS knowledge for migration and legacy analysis. Azure Function Apps and Logic Apps.
• HR, identity governance, procurement or ERP (JD Edwards) integration experience.
• Microsoft Fabric exposure.
• Databricks Certified Data Engineer Associate, DP-700 (or the retired DP-203), AZ-104, AZ-400, ITIL 4 Foundation.

5. Key relationships

HR, identity, procurement and finance system owners: Feed validation, business impact and data-correction approval

SaaS and data vendors (e.g. Coupa, Fieldglass, fuel card providers): API and file contract changes, authentication and vendor escalation

Data warehouse, Power BI and Azure Integration engineers: Shared dependencies, migration and cross-platform triage

Service Desk / L1: Intake, initial triage, shift-left knowledge articles and incident escalation

Technical Platform Owner / Principal Analyst: Platform priorities, risks, technical governance and acceptance

Change and release management: CAB approval, controlled releases and production validation

Infrastructure, DBA and Cyber Security teams: Servers, databases, privileged access, certificates and security approvals

6. Performance measures

• Incident response and resolution against agreed service levels and business-critical processing windows.
• Completion and effectiveness of proactive monitoring, exception detection and recovery.
• Service request timeliness and first-time-right fulfilment.
• Changes delivered successfully through change control, without unplanned outage or rollback.
• Reduction in repeat incidents through problem records, root cause analysis and permanent fixes.
• Accuracy and currency of runbooks, knowledge articles, ServiceNow resolution notes and operational documentation.
• Stakeholder satisfaction and timely communication during incidents and changes.
• Migration backlog progressed with recorded owner, status and decommissioning of retired SSIS packages.

7. Transition readiness (by 21 December 2026)

Independence criteria Fujitsu must demonstrate before taking over from the incumbent provider. Remove this section after transition acceptance.

• Daily ADF and Databricks monitoring run without incumbent assistance. Alert recipients repointed to Fujitsu groups.
• Resolve at least two pipeline failures unaided, including a rerun or backfill.
• Deliver at least one pipeline or notebook enhancement end to end through Azure DevOps and change control before the Christmas change freeze.
• Integration runtimes, linked services, secrets and DevOps access under Fujitsu control. Credential and certificate expiry register handed over.
• Integration backlog (remaining SSIS migrations, stalled changes) held with owner and status, and the migration completion decision recorded.
• Incident SLAs met for four consecutive weeks.

8. Working arrangements and cross-cover

Provide coverage of Australian business hours (AEST/AEDT) through agreed shift patterns, including the Australian morning window. Participate in a rostered on-call arrangement outside business hours, including weekends, for incidents and business-critical processing windows (e.g. identity, procurement and finance feed deadlines and ERP outage windows), and in planned after-hours changes and maintenance. All operational activities must follow approved procedures, access controls, Cleanaway change governance and delegated authority.

Cross-cover: Mutual cover with the data warehouse engineer (SSIS migration, data warehouse loads and pipelines feeding the data warehouse).

At Fujitsu, we are committed to an inclusive recruitment process that values the diverse backgrounds and experiences of all applicants. We believe that hiring people from a wide variety of backgrounds makes us stronger, not because it's the right thing to do, but because it allows us to draw on a wider range of perspectives and life experiences.