Senior Data Engineer

Job Description Overview:

• Designs, develops, tests, deploys and supports complex data pipelines, data products and data management solutions across modern cloud, warehouse and lakehouse platforms. Applies software-engineering, data-modeling and distributed-processing practices to deliver secure, reliable, maintainable and cost-conscious solutions. Works independently on defined outcomes, collaborates across technology and data teams, and contributes to reusable engineering patterns and continuous improvement.

Primary Responsibilities:

• Design, develop, test, deploy and support complex production data pipelines and data products with guidance on broader architecture where needed.
• Develop maintainable data-processing and transformation solutions using SQL and programming languages such as Python.
• Implement solutions using Snowflake, Databricks, dbt and comparable modern data platforms and frameworks.
• Build data solutions across relational databases, cloud object storage, warehouse and lakehouse architectures, including open table formats such as Apache Iceberg where applicable.
• Develop batch, micro-batch, CDC and streaming pipelines using approved enterprise patterns and technologies such as Apache Kafka or comparable platforms.
• Integrate data from relational and mainframe systems, APIs, event streams, SaaS platforms, files and object storage.
• Apply normalized, dimensional and denormalized data-modeling techniques appropriate to analytical and data-product requirements.
• Tune SQL, Snowflake and Databricks/Spark workloads for performance and cost using appropriate partitioning, parallelism and compute practices.
• Implement data quality, reconciliation, metadata, lineage, logging, monitoring and alerting controls.
• Build pipelines that support error handling, restartability, idempotency, retry, replay and recovery using established patterns.
• Participate in production support, incident resolution and root-cause analysis; implement corrective actions to prevent recurrence.
• Use Git, peer review, automated testing, CI/CD and environment promotion practices for data engineering delivery.
• Develop and reuse shared components, templates and patterns; contribute improvements to engineering standards and documentation.
• Participate in technical design and code reviews, identify implementation risks and escalate architectural concerns appropriately.
• Partner with product owners, architects, governance, cybersecurity, platform and consuming teams to clarify requirements and deliver supported solutions.
• Contribute to technical epics and stories, estimate assigned work, identify dependencies and deliver through Agile practices.
• Provide day-to-day technical assistance and informal mentoring to less-experienced engineers. This role does not have direct people-management responsibility.
• Understand and adhere to the Company’s risk and regulatory standards, policies and controls in accordance with the Company’s Risk Appetite.
• Design, implement, maintain and enhance internal controls to mitigate risk on an ongoing basis. Identify risk-related issues requiring escalation to management.
• Maintain internal control standards, including timely implementation of internal and external audit points and issues raised by external regulators, as applicable.
• Promote an environment that supports belonging and reflects the Company brand.
• Complete other related duties as assigned.

Education and Experience Required:

• Associate’s degree and a minimum of 5 years’ systems analysis and/ or application development work experience or Bachelor's degree and a minimum of 3 years’ systems analysis and/ or application development work experience. In lieu of degree, a combined minimum of 7 years’ education and/or relevant work experience, including a minimum of 3 years’ systems analysis and/or application development work experience
• Strong SQL skills, including development of complex transformations and analytical queries.
• Experience with Python or another general-purpose programming language used for data engineering.
• Hands-on experience with Snowflake, Databricks or a comparable cloud data platform.
• Experience with ELT/ETL and transformation practices, including dbt or a comparable framework.
• Experience with relational data stores and warehouse or lakehouse data-modeling concepts.
• Experience with distributed processing using Apache Spark/PySpark or a comparable framework.
• Experience with Git-based source control, automated testing and CI/CD fundamentals.
• Experience with batch data processing and working knowledge of streaming or event-driven concepts.
• Knowledge of cloud and object-storage concepts, preferably in Microsoft Azure.
• Experience working with structured and semi-structured data using formats such as Parquet, JSON, CSV, Avro and XML.
• Experience with Linux/Unix environments and scripting.
• Demonstrated ability to independently deliver complex data pipeline components and data products within an established architecture.
• Demonstrated ability to troubleshoot data-processing, data-quality and production issues across multiple technologies.
• Demonstrated ability to apply established engineering patterns while identifying opportunities for improvement.
• Demonstrated ability to communicate implementation choices, risks and dependencies to technical and product partners.
• Demonstrated ability to manage assigned work from design through production support with limited day-to-day supervision.

Education and Experience Preferred:

• Production experience with Snowflake, Databricks, Apache Spark/PySpark and dbt.
• Experience with Apache Iceberg or another open table format.
• Experience with Apache Kafka, CDC or event-driven data processing.
• Experience with orchestration technologies such as Apache Airflow, Azure Data Factory, Databricks Workflows or dbt orchestration.
• Experience with Terraform or another Infrastructure as Code technology.
• Experience with data catalogs, metadata, lineage, observability or automated data-quality tooling.
• Experience delivering governed enterprise data products.
• Experience working in a regulated environment such as financial services.

Location