Grigorii Zavyalov

Designed a ClickHouse data structure for storing and aggregating Spotify track-level information. Implemented an ETL workflow in Python for cleaning and loading the dataset, then created interactive Streamlit dashboards for exploration of audio characteristics and popular artists. Applied machine learning methods (KMeans and PCA) using scikit-learn to cluster and analyze the dataset.

Built a local streaming pipeline that generates JSON events and processes them with PySpark Structured Streaming. Implemented transformations, filtering, aggregations, duplicate checks, and required-field validation. Stored intermediate results as Parquet in S3/MinIO and loaded final outputs into ClickHouse.

Designed and implemented automated ETL workflows in Python and SQL for a medical monitoring system, including data loading, transformations, and reporting outputs. Built scheduled data pipelines using Prefect and added end-to-end monitoring with logging and task status tracking. Integrated Prefect workflows with ClickHouse and external REST APIs, and containerized services with Docker/Docker Compose while optimizing SQL performance to reduce runtime of batch procedures.

Designed and delivered analytical data marts for internal systems and city digital services. Developed complex SQL for ClickHouse and PostgreSQL and performed query optimizations to speed up report and export generation. Automated recurring data loading and processing with Apache Airflow, implemented data backup/export storage to MinIO (S3-compatible), and established data quality checks for completeness and consistency using Agile team practices with Git and Jira.