Data Engineer – PySpark

Highbrow Technology Inc is seeking a Data Engineer to design, develop, and maintain scalable data engineering solutions. The role focuses on PySpark-based data pipelines, ETL/ELT workflows, distributed data processing, and support for data lakes and data warehouses.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and Python
• Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data
• Develop data transformation and data processing solutions using Apache Spark
• Work with distributed data processing frameworks and big data technologies
• Perform data cleansing, validation, integration, and transformation
• Optimize Spark jobs for performance, scalability, and reliability
• Collaborate with data architects, analysts, and other engineering teams
• Troubleshoot data pipeline issues and ensure data quality and consistency
• Support data lake and data warehouse solutions

Skills

• 6+ years of experience in Data Engineering or related roles
• Strong hands-on experience with PySpark and Python
• Strong experience with Apache Spark and distributed data processing
• Experience developing and maintaining ETL/ELT data pipelines
• Strong SQL skills and experience working with large datasets
• Knowledge of data warehousing and data lake concepts
• Experience with data transformation, optimization, and performance tuning
• Ability to work independently and collaboratively in a hybrid work environment
• Visa-independent work authorization
• Experience with cloud platforms such as AWS, Azure, or GCP
• Knowledge of Databricks, Delta Lake, or similar data platforms
• Experience with Snowflake or other cloud data warehouses
• Familiarity with Airflow or other workflow orchestration tools
• Experience with CI/CD and data engineering best practices

Benefits

• Hybrid – Onsite

Company Overview