Data Engineer

Responsibilities:

• Data Requirements and Reporting. Deliver data requirements raised by internal and external stakeholders, including regulatory, finance, banking, compliance, sales, business development, operations, core processing, and SRE teams. Automate recurring requirements to reduce manual effort.
• Lambda Development. Build and maintain AWS Lambda functions in Python and Java for scheduled reports, event-driven processing, and ad-hoc data extractions. Integrate Lambdas with S3 DynamoDB, Aurora PostgreSQL, SQS, and EventBridge.
• Athena and SQL. Author and optimize Athena (Presto / Trino) queries against the datalake. Manage partition strategies and joins across merchant, onboarding, transaction, settlement, and reconciliation datasets.
• Reporting Pipelines. Build and maintain daily, weekly, and monthly reporting pipelines that feed operations, finance, sales, and regulatory workflows.
• Dashboards. Develop and maintain QuickSight dashboards for payment success rate analysis, liability monitoring, settlement tracking, and merchant performance, including custom SQL data sources and KPI tracking.
• Merchant Monitoring. Build and maintain merchant-monitoring pipelines and dashboards covering PSR trends, transaction volume and count anomalies, settlement timeliness, liability exposure, and dormancy. Investigate PSR drops at the merchant level and surface findings to Operations, Risk, and Account Management.
• Data Modeling. Design DynamoDB schemas, Aurora table structures, and Glue Catalog DDLs for new data domains. Map nested JSON structures into Athena-compatible schemas.
• Cross-team Collaboration. Work with Engineering, Risk, Banking, Operations, and Compliance to translate business requirements into data deliverables.
• Operational Support. Debug failing Lambdas, investigate data quality issues, and resolve constraint violations and pipeline errors. Maintain CloudWatch alerts and monitoring.

Requirements:

• Python. Python 3 x dataclasses, type hints, comprehensions, context managers, JSON and CSV processing, error handling, and AWS Lambda handlers using boto3
• Java. Java 17 Spring Boot, Lombok, AWS Lambda Java runtimes, dependency injection, command-line runners, and OpenCSV.
• SQL and Athena. Joins, CTEs, window functions, UNIONs, partition filters, and date functions. Familiarity with Presto / Trino syntax (date_parse, from_iso8601_timestamp, date_format).
• AWS Services. Hands-on experience with Lambda, S3 DynamoDB, Aurora PostgreSQL, Athena, Glue, SQS, EventBridge, IAM, and CloudWatch.
• Data Engineering Fundamentals. Partitioning, batch and streaming patterns, ETL design, schema evolution, and data quality controls.
• Tooling. AWS CLI, Git, Maven, IntelliJ / PyCharm, and command-line workflows.
• Technology Stack Layer: Technologies, Languages: Python 3 x, Java 17 (Spring Boot), SQL Compute AWS Lambda (Python and Java runtimes), EventBridge schedules, SQS triggers Storage AWS S3 DynamoDB, Aurora PostgreSQL, RDS Query and Analytics Athena (Presto / Trino), Glue Catalog, AWS Data Lake Visualization Amazon QuickSight Messaging SQS, SNS, DynamoDB Streams; Tooling AWS CLI, Git, Maven, IntelliJ IDEA, PyCharm, CloudWatch; Frameworks Spring Boot (for Lambda), boto3 psycopg2 OpenCSV, Lombok.

Internal Preferred Skills:

• Experience in the payments domain (acquirers, issuers, FIRC, settlements, MDR, MCC, TID config, 3DS authentication).
• Experience integrating with notification systems (SQS-driven email pipelines, Thymeleaf templates).
• Familiarity with QuickSight (dashboards, custom SQL, calculated fields, parameters).
• Experience with DynamoDB GSIs and query optimization.
• Exposure to schema mappers, JSON-to-CSV flatteners, and ingestion frameworks.
• Understanding of GCM-encrypted DynamoDB items and DEK-based decryption patterns.

Skills

AWS, Athena, DynamoDB, Java, Lambda, Python, SQL