Data Engineer/BI Developer-DA&A

Job Title

Data Engineer / BI Developer - DA&A

Expected Work Experience

3 to 6 Years

Job Location

India

Job Summary

Design and implement reusable, parameterized, secure, and idempotent onboarding modules and orchestration that coordinates them for data and analytics platforms. The engineer will consolidate existing automation, harden fragile components, expose manually triggered scripts, and automate remaining manual steps across application scaffolding, cloud resources, and Release Workflow integration, with a focus on Data Automation & AI and AWS DevOps tools such as X-Ray. Candidate must have relevant experience of 3 to 6 Years.

Key Responsibilities

• Design and implement reusable, parameterized, secure, and idempotent onboarding modules and the orchestration that coordinates them across data engineering and BI workflows.
• Consolidate existing automation, harden fragile components, expose manually triggered scripts, and automate remaining manual steps across application scaffolding, cloud resources, and Release Workflow integration.
• Deliver reusable application-scaffolding and cloud-resource modules with automated tests tailored to data pipelines, BI solutions, and analytics platforms.
• Implement Jenkins orchestration with parallel NonProd execution, state management, retries, resume capabilities, timing, and audit for data and BI onboarding flows.
• Integrate Release Workflow for request, trigger, status, approvals, and evidence to support end-to-end onboarding of data and analytics applications.
• Provide diagnostics, telemetry, dashboard feeds, outcome summary data, documentation, and handover materials for stakeholders in DA&A.

Activity discovery and automation triage

• Partner with SMEs to document current one-time and per-environment activities, dependencies, inputs, outputs, approval points, and failure modes for data pipelines and BI solutions.
• Assess existing scripts and pipelines for reuse; classify them as wrap, harden, expose, build, or human decision.
• Create proof points early to validate that Release Workflow can invoke Jenkins or approved APIs and display run status for onboarding runs.

Reusable module engineering

• Build application-scaffolding automation for repository creation from template, starter application setup with health check, configuration, and developer access for data and BI projects.
• Integrate and harden existing Terraform modules for cloud resources while preserving the existing human cloud-approval step.
• Create environment-parameterized modules driven by one configuration matrix rather than duplicated environment-specific code.
• Implement completion validation and automated tests for each module; ensure reruns do not create duplicate or conflicting resources.
• Use secure patterns for credentials, secrets, identity, least privilege, sensitive logging, and approved tool usage across data and analytics environments.

Orchestration and workflow integration

• Implement the Jenkins onboarding orchestrator, dependency sequencing, run identifiers, state persistence, retries, resume from the failed step, and audit trail.
• Run one-time activities first, execute five NonProd environments in parallel, pause for production approval, and then coordinate Canary, Central, and East deployments where applicable.
• Integrate the Release Workflow request form, trigger, status, cloud approval, production approval evidence, and outcome information.
• Generate structured outcome data for dashboards and summary emails, including resources, links, results, failures, and recommended follow-ups.

Engineering quality and transition

• Write unit and component tests, support integration and failure-injection testing, remediate defects, and participate in peer reviews.
• Document modules, configuration, dependencies, troubleshooting steps, operational controls, and known failure modes.
• Contribute to weekly demos, gate evidence, runbooks, knowledge-transfer sessions, code handover, launch, and warranty defect resolution.

Other Capabilities

• Ability to work independently in a customer environment and collaborate effectively with distributed US and India stakeholders.
• Strong focus on engineering telemetry, observability, and auditability for data and BI onboarding processes.
• Comfortable working across Data Automation & AI initiatives and AWS DevOps tooling, including X-Ray, in complex enterprise environments.
• Strong problem-solving skills with the ability to triage issues across pipelines, infrastructure, and workflow integrations.
• Clear communication skills for explaining automation designs, onboarding flows, and operational impacts to technical and non-technical stakeholders.

Qualifications and Skills

• 5-9 years in DevOps, cloud automation, platform engineering, or site reliability engineering.
• Hands-on delivery experience with Jenkins pipelines and shared libraries, Terraform, Azure/AKS, REST APIs, scripting, and Git-based workflows.
• Experience building production-grade automation with tests, logging, error handling, retries, state management, and rollback or recovery strategies.
• Bachelor’s degree in a technical discipline or equivalent practical experience.
• Mandatory technical competency in advanced Jenkins pipeline development using declarative and scripted pipelines, shared libraries, credentials, parameters, artifacts, and reusable templates.
• Strong Terraform module integration, remote-state and state-safety concepts, plan/apply controls, approval gates, drift awareness, and idempotent provisioning.
• Azure and AKS fundamentals: resource hierarchy, identity and RBAC, networking awareness, Kubernetes resources, namespaces, service accounts, ingress and configuration, and health probes.
• Proficiency in Python, PowerShell, Bash, or Groovy for API integration and automation wrappers.
• API integration for request ingestion, status updates, evidence exchange, and orchestration triggers.
• Engineering telemetry skills: structured logs, run IDs, step timings, failure classification, actionable diagnostics, and audit records.
• Experience with enterprise release workflow tools, ServiceNow-style integrations, or custom developer portals is preferred.
• Experience creating repository templates, starter applications, access automation, and configuration generators is an advantage.
• Azure DevOps Engineer Expert, Terraform Associate, CKA/CKAD, or comparable certification is desirable.
• Experience with contract and schema validation, policy-as-code, secrets management, or automated evidence packs is beneficial.
• Exposure to Data Automation & AI and AWS DevOps & Developer Tools such as X-Ray is highly valued.