Data Engineer (ETL/ELT) (Finance Domain)

Key Responsibilities:

• Develop and maintain data pipelines and ETL/ELT/analytics engineering workflows to support advanced search and retrieval capabilities.
• Collaborate with data teams to understand requirements and automate deployment and monitoring.
• Optimize data storage and troubleshoot issues to enhance performance.
• Work closely with data engineers and relevant parties to ensure sprints are planned out properly and completed timely.
• Perform BAU monitoring, investigations and resolutions.
• Assist with data governance and data management initiatives.
• Production and application support from a day-to-day basis.

Skillset Requirements

• Possess a degree in Computer Science/Information Technology or related fields.
• At least 5 years of hands-on experience in software or data engineering with proficiency in Python.
• Strong experience in unit and integration testing.
• Familiarity with DevOps practices and Agile methodologies.
• Experience with AWS and Kubernetes (K8s).
• Familiarity with data platforms such as Snowflake, Databricks, Apache Spark, Apache Hive, open table formats (Delta Lake, Apache Iceberg), and vector databases.
• Experience with orchestration tools such as Apache Airflow, Dagster, Prefect, and Temporal.
• Familiarity with GitHub workflows and Datadog.
• Good written and verbal communication skills.
• Agile, fast learner and able to adapt to changes.

Skills

Written and Verbal Communication Abilities, Apache Spark, Kubernetes, Database Systems, maximize storage capacity, Data Governance, Workflow Development, Computer Science, Production Support, Workflow Orchestration, Integration Testing, Deployment Automation, Workflow Solutions, Agile & Waterfall Methodologies, Sprint methodology, Bau Support