Senior Data Engineer

United Network for Organ Sharing (UNOS) is seeking a Senior Data Engineer to lead the design and implementation of end-to-end data solutions in an Azure-based cloud environment. The role focuses on building scalable data pipelines and lakehouse architectures, ensuring data quality and reliability, optimizing data platforms, collaborating with cross-functional teams, and mentoring junior engineers.

Responsibilities

• Architect and implement secure, scalable data pipelines using Azure Data Factory, Azure Functions, and Azure Data Lake Storage
• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks (PySpark/Spark SQL) to ingest, transform, and process large volumes of structured and unstructured data
• Build and manage data pipelines using Azure services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure Event Hubs, and Azure SQL Database
• Implement data quality, validation, and monitoring frameworks to ensure reliability and accuracy of data pipelines
• Collaborate cross-functionally with data scientists, analysts, and business stakeholders to understand data requirements and deliver fit-for-purpose data solutions
• Lead modernization and optimization efforts, improving pipeline performance, maintainability, and scalability
• Support CI/CD practices for data pipelines using tools such as Azure DevOps, Git, and Databricks Repos
• Detect and resolve data quality issues; implement automated audits and monitoring processes
• Troubleshoot and resolve production data pipeline issues, ensuring high availability and minimal downtime
• Act as a technical leader on schema design, performance tuning, and Azure data architecture
• Mentor and support junior engineers across data engineering, analytics, and BI teams
• Participate in and lead code reviews, promoting clean, well-documented, and testable code
• Stay current on trends in data engineering and cloud technologies, identifying opportunities to innovate

Skills

• * 8+ years of hands-on experience in data engineering, including designing and implementing enterprise-scale data solutions
• * 2+ years of experience developing and operating Azure cloud-native data platforms
• * Expertise in MS SQL Server, Python (pandas, PySpark), Azure Data Factory, Azure Functions and Azure Data Lake Storage
• * Strong expertise with Azure Databricks, including Spark (PySpark/Scala), Delta Lake, and cluster/job optimization
• * Solid understanding and hands-on experience building Data Lakehouse architecture
• * Experience working with a variety of file formats (e.g., CSV, JSON, XML, Parquet)
• * Experience with version control (Git) and CI/CD pipelines for data engineering workflows
• * Familiarity using REST APIs for data extraction and integration
• * Proven experience designing and implementing data solutions
• * Strong understanding of cloud architecture, data warehousing and modern data stack components
• * Demonstrated ability to perform root cause analysis on data and processing issues
• * Strong problem-solving skills with the ability to explain technical concepts to non-technical audiences
• * A successful history of manipulating, processing and extracting value from large disparate datasets
• * Experience with Big Data technologies such as Databricks, Spark, or Azure Synapse
• * Knowledge of CI/CD workflows, version control, and agile development practices
• * Familiarity with data governance, privacy, and compliance frameworks
• * Experience with data warehousing, analytics tools, and BI platforms
• * Familiarity with streaming data technologies (Azure Event Hubs, Kafka, Structured Streaming)
• * 4-year degree in computer science, engineering or other related IT field of study, or equivalent professional work experience
• * Prolonged periods of sitting at a desk and working on a computer
• * Frequent reaching, handling, and fine manipulation for using office equipment, filing, and managing paperwork
• * Manual dexterity sufficient to operate a keyboard, mouse, and other office tools
• * Occasional standing, walking, and bending
• * Ability to lift up to 10-20 pounds occasionally
• * Vision abilities required include close vision for computer work and reading documents

Company Overview