Staff Data Engineer

Unite USA Inc. dba Unite Us

Staff Data Engineer

Location: New York, New York

Job Description

The Staff Data Engineer will lead enterprise-wide data architecture strategy and long-term data

platform design. Evaluate and approve major technical designs, including establishing

engineering best practices. Serve as technical advisor to leadership and influence product/data

roadmap. Champion data governance, democratization, and scalable data standards. Work with

cross functional teams including the Solutions Delivery, Business Intelligence; Predictive

Analytics, and Enterprise Services teams to assist with and support any data-related technical

issues and/or needs. Analyze, manipulate, and process large sets of data using statistical

software. Develop ETL (Extract, Transform, Load) and data pipeline solutions using Airflow,

DBT, Python, and Snowflake to load a data warehouse. Design and manage ETL workflows to

support large-scale ingestion, transformation, and loading of data. Build and scale CI/CD

pipelines and enterprise-wide data quality frameworks. Test internal data pipelines for reliability

and performance. Test, validate, and reformulate models to ensure accuracy. Complete and

improve monitoring and validation processes to maintain pipeline reliability and data accuracy.

Write new functions or applications in programming languages to conduct analysis. Build

streaming pipelines with Confluent Kafka for event-driven processing. Build dashboards and

monitors for pipeline health, freshness, uptime, and SLA compliance. Apply data modeling to

extract and analyze information from large structured and unstructured datasets. Telecommuting

available anywhere in the United States. Annual salary between $175,000 - $200,000.

Job Requirements

Requires at least a Master’s degree or foreign equivalent in Data Science, Applied Computer

Science, Computer Science or a related field. Must possess 4 years (48 months) of experience

with all of the following: (a) developing and optimizing scalable pipelines using Apache Airflow

and Python; (b) building streaming pipelines using Kafka; (c) tuning big data systems using

Kafka and Hive; (d) using Spark to create jobs for batch and streaming analytics; (e) writing and

using complex SQL queries to retrieve data from data warehouses for analysis and reporting (f)

using AWS for data processing and large-volume workloads; (g) automating deployment of code

and processes using Python; (h) working with cloud-based infrastructure; (i) building Datadog

dashboards to monitor pipelines and system performance; (j) using Docker and Kubernetes to

improve and validate containerization and automation reliability; (k) implementing Continuous

Integration/Continuous Deployment (CI/CD) using Jenkins; and (l) utilizing all of the following:

GitHub as a code repository, Windows and Linux operating systems. Experience may be gained

concurrently. Telecommuting available anywhere in the United States.

Benefits: Paid Parental Leave; 401K Match; Flexible Vacation Time; Medical, Dental and

Vision Coverage; Commuter Benefits.