Data Engineering Specialist - Hadoop & NiFi
First seen by Alion on Nov 13, 2026.
Role Summary:
We are looking for an experienced Data Engineering Specialist to design, develop, and optimize scalable data ingestion and ETL pipelines across the Hadoop ecosystem. The ideal candidate should have strong hands-on experience with Apache NiFi, Spark/PySpark, Hadoop/HDFS, Apache Airflow, Cloudera Data Engineering (CDE), Hive, SQL, and Python.
Mandatory Skills:
- Apache NiFi
- Spark / PySpark
- Hadoop / HDFS
- Apache Airflow
- Cloudera Data Engineering (CDE)
- Hive & SQL
- Python
Key Responsibilities:
- Design, develop, and maintain scalable ETL and data ingestion pipelines.
- Build and optimize data processing solutions using Spark, PySpark, and NiFi.
- Develop and manage Airflow workflows for scheduling and orchestration.
- Work with Hadoop, HDFS, Hive, and CDE for enterprise data processing.
- Write efficient Hive SQL and Python code for data transformation and automation.
- Monitor, troubleshoot, and optimize production data pipelines.
- Collaborate with cross-functional teams to deliver reliable data solutions.
Required Qualifications:
- 8+ years of experience in Data Engineering and Big Data technologies.
- Strong expertise in NiFi, Spark/PySpark, Hive SQL, Python, and Airflow.
- Good understanding of Hadoop, HDFS, distributed data processing, and ETL best practices.
- Hands-on experience with Cloudera Data Engineering (CDE) or similar enterprise data platforms.
- Experience supporting and optimizing production data pipelines.
- Strong analytical, problem-solving, and communication skills.
Skills
Data Engineering, Apache NiFi, Spark, Python, Apache Airflow, Hadoop, Hive, SQL, Big Data, HDFS, ETL