Derick Juma

Data Engineering Manager

SeniorAI/ML Engineer

8/10Data Pipeline & Feature Engineering

6repositories checked by code analysis

4projects

Overview

Senior-level data engineer specializing in end-to-end ETL and ingestion pipelines for environmental and agricultural data. The strongest proven skill is building robust S3-backed ingestion and backfill systems with concurrency, retries and error handling as implemented in src/ingestion/openaq_to_bronze.py and src/ingestion/backfill_bronze.py. Limited evidence exists for advanced statistical modeling, production MLOps, CI/CD and formal data versioning workflows in public code.

Phone

Projects

InVoice - Voice-to-Invoice Application

Developed and published a mobile app that converts voice input into structured invoice content. Implemented a Python backend with REST endpoints that generate PDF invoices and validate extracted billing items, quantities, and prices via JSON. Packaged the client app as a Google Play application for end-user use.

Retail Market Expansion SQL Analytics

Performed analytical SQL work to rank coffee retail markets by revenue, rent efficiency, and month-over-month growth. Used CTEs and window functions such as DENSE_RANK and LAG, along with multi-table joins to compute the required metrics. Focused on producing structured ranking outputs for comparisons across Indian cities.

Pan-Africa Air Quality Pipeline

Designed a batch pipeline using a medallion approach to ingest air quality readings from archived data into cloud storage. Implemented filtering and idempotent daily ingestion logic with an Airflow DAG, including Bronze/Silver/Gold bucket structuring. Containerized the orchestration and Spark runtime with Docker Compose and provisioned the storage buckets using Terraform.

Kenya Agri-Climate Data Pipeline

Built an end-to-end batch data pipeline to pull multi-year climate and agricultural market data from external APIs and land raw files in cloud storage. Orchestrated ingestion and loading into BigQuery, and created a dbt star-schema model with staging, marts, and rolling weather aggregations. Added data validation via dbt tests and used Terraform for infrastructure provisioning plus containerized orchestration for repeatable runs.

Technical skills

Languages

Kotlin

Dart

Python

SQL

Mobile

Jetpack Compose

Flutter

Lottie

DevOps

AWS

GCP

Amazon S3

Docker Compose

Other

Boto3

Google BigQuery

Airflow

Spark

PostgreSQL

GitHub

GitHub Actions

dbt

Docker

Git

Linux

Terraform

Rest API

pySpark

State Management

Declarative UI

Material Design

ETL/ELT

Senior AI/ML Engineer Confidence: High Data-centric

Data engineering practitioner focused on building production-ready ETL and ingestion pipelines for environmental and agricultural datasets, operating at a solid middle-to-senior level. The strongest proven skill is engineering robust data ingestion and S3-based bronze-layer pipelines, evidenced by the openaq_to_bronze.py and backfill_bronze.py implementations with pagination, retry handling and threaded backfill orchestration. There is little to no public evidence of model training, experiment tracking or novel ML research in the available code.

Model Architecture & Training

How well models are designed and trained

Not evidenced in public code

Data Pipeline & Feature Engineering

8/10

How data is prepared for models

Strong, production-oriented data pipeline engineering: clear ingestion phases (discovery, bronze ingestion, backfill), S3 key design, pagination and rate-limit handling, and repeatable metadata workflows.

Evidence

jude-craft/pan-africa-air-quality/src/ingestion/openaq_to_bronze.py: build_source_key / build_destination_key / ingest_date jude-craft/pan-africa-air-quality/src/ingestion/backfill_bronze.py: run_backfill with ThreadPoolExecutor and per-thread make_s3_client jude-craft/pan-africa-air-quality/src/ingestion/fetch_locations.py: fetch_locations_for_country with pagination, rate-limit handling, and flatten_location

Experimentation & Evaluation

2/10

How results are measured and tested

Lightweight verification and sanity checks are present (infrastructure connectivity tests and summary printing) but no formal experiment tracking, evaluation pipelines, or reproducible ML experiments.

Evidence

jude-craft/pan-africa-air-quality/tests/test_aws_connectivity.py: verify_s3_buckets jude-craft/pan-africa-air-quality/src/ingestion/fetch_locations.py: print_summary (sanity checks and summaries)

MLOps & Deployment

4/10

How models are shipped to production

Operational pipeline pieces and orchestration are present (Airflow DAGs, CLI-style ingestion scripts, dotenv usage, docker-compose and pyproject metadata) indicating practical deployment experience but not full MLOps/model lifecycle tooling.

Evidence

jude-craft/pan-africa-air-quality/dags/bronze_ingestion_dag.py: bronze_ingestion_dag (Airflow DAG) jude-craft/pan-africa-air-quality/pyproject.toml: Apache Airflow in dependencies and project script entrypoint jude-craft/pan-africa-air-quality/src/ingestion/openaq_to_bronze.py: CLI entrypoint and default_ingestion_date orchestration

Computational Efficiency

4/10

How efficiently computing resources are used

Some efficiency and robustness work is evident: use of paginators, adaptive boto3 retry configs, ThreadPoolExecutor for parallel backfills and polite rate-limit sleeps; however, there is limited evidence of advanced GPU/compute optimizations or profiling.

Evidence

jude-craft/pan-africa-air-quality/src/ingestion/backfill_bronze.py: ThreadPoolExecutor usage and Config(retries={'max_attempts': 5, 'mode': 'adaptive'}) jude-craft/pan-africa-air-quality/src/ingestion/fetch_locations.py: PAGE_LIMIT pagination loop and rate-limit handling with Retry-After waits jude-craft/pan-africa-air-quality/src/ingestion/openaq_to_bronze.py: use of S3 paginator and destination_exists head_object pattern

Research Depth & Innovation

Depth of research and new ideas

Not evidenced in public code

Verified artifacts

Android-Community-MUST/kotlin-hustlehub· ★ 7 NovaStackv1/Agrisense· ★ 2 NATO-dotcom/ClinicManagement_-JavaDB_Connectivity· contrib NATO-dotcom/Retail-Sales-Analysis-SQL-Project· contrib

Expertise

MLOps & Model Lifecycle· Middle

Industries

Data & Analytics· Senior

Farming & Agriculture· Middle

Technologies

SQL

PostgreSQL

Terraform

GCP

Docker Compose

GitHub Actions

Git

AWS

Docker

pySpark

GitHub

Amazon S3

Linux

Databricks· mentioned only

dbt· mentioned only

pySpark· mentioned only

SQL· mentioned only

Recommendations

Develop end-to-end ML experiment tracking and evaluation pipelines (MLflow or Weights & Biases) to support future modeling work on the cleaned datasets.
Add automated unit and integration tests for ingestion logic (e.g., mocking S3 and HTTP responses) and CI/CD configuration to validate deployments.
Introduce configurable retry/backoff wrappers and centralized logging/metrics (structured logs, Prometheus/Grafana) to improve observability and incident response.
Harden network operations: add explicit retry/backoff for market_extract and weather_extract requests, and circuit-breaker patterns for unstable upstreams.

Repositories

The developer's experience in this domain has been verified based on AI analysis of the following repositories:

kenya-agri-climate-data-pipeline pyspark-engineering-lab pan-africa-air-quality

Senior Data Scientist Confidence: High Data Engineer

Statistical Rigor

1/10

Correct use of statistics

Minimal statistical rigor or formal hypothesis testing; SQL analyses are diagnostic but do not include uncertainty quantification or causal inference.

Evidence

kenya_agri_dbt/analyses/audit_missing_weather_days.sql

Data Wrangling & Cleaning

8/10

Preparing and cleaning data

Strong data wrangling and cleaning practices across extraction, schema checks, pagination, rate-limit handling, retry logic, and classification of active sources; clear attention to preventing wasted calls and handling missing sources.

Evidence

extract/market_extract.py

src/ingestion/openaq_to_bronze.py

src/ingestion/backfill_bronze.py

Exploratory Analysis & Visualization

3/10

Exploring and visualizing data

Exploratory work exists mainly as instructional PySpark notebooks and CTE-based dbt queries; displays modular queries and basic data storytelling but lacks polished visualizations or written interpretation of analytical findings.

Evidence

pyspark-engineering-lab/lessons/09-spark-sql/spark_sql.py

pyspark-engineering-lab/lessons/01-spark-fundamentals/spark_fundamentals.py

Predictive Modeling

1/10

Building models that predict

Little to no predictive modeling evidence; repository focuses on ingestion, transformation and analytics-ready tables rather than model development or evaluation.

Evidence

pyspark-engineering-lab/lessons/09-spark-sql/spark_sql.py

Business Insight & Impact

5/10

Turning analysis into business value

Reasonable linking of analytics to business questions such as matching producer climate metrics to consumer market prices and seeding trade routes; good domain framing but limited explicit business-metric tradeoff analysis or cost-sensitive error reasoning.

Evidence

analytics/kenya_agri_dbt/models/marts/fct_climate_market_impact.sql

extract/market_extract.py

Reproducibility & Notebook Hygiene

4/10

Clean, repeatable analysis

Scripts use dotenv, __main__ guards, basic tests for AWS connectivity, CLI args and retry configs; reproducibility is present at script level but there is limited evidence of pinned environments, data versioning, CI pipelines or DVC-style artifact management.

Evidence

src/ingestion/openaq_to_bronze.py

tests/test_aws_connectivity.py

Expertise

Big Data· Senior

Data Warehouse· Middle

Industries

Data & Analytics· Senior

Farming & Agriculture· Middle

Technologies

Python· Senior

Spark

Airflow

dbt

Boto3

Google BigQuery

Databricks· mentioned only

pySpark· mentioned only

SQL· mentioned only

Recommendations

Use to build and harden production data ingestion and backfill systems on AWS or GCS with PySpark and Airflow, including ownerable S3 key schemas and retry logic.
Develop dbt-based analytics layers that join cleaned climate and market feeds into business-facing fact tables and seeded trade-route dimensions.
Instrument pipelines with monitoring, alerting and observability (dataplane metrics, error budgets, runbook docs) and add CI/CD and data-versioning (DVC or equivalent) for reproducibility.

Repositories

The developer's experience in this domain has been verified based on AI analysis of the following repositories:

kenya-agri-climate-data-pipeline pyspark-engineering-lab pan-africa-air-quality

Middle Mobile Developer Confidence: High Cross-platform

Flutter developer (Middle) focused on polished cross-platform mobile UI and small-business flows. Proven strength is building production-feeling Flutter user interfaces and stateful screens, for example ShopMate/lib/src/features/screens/sales/sell_screen.dart and ShopMate/lib/src/features/screens/reports/reports_screen.dart. Public code lacks evidence of background sync/conflict resolution, advanced native device integrations, formal performance measurement or CI/release engineering.

Platform Native Mastery

3/10

Knowing the mobile platform

Basic platform lifecycle awareness is present (dispose, post-frame callbacks, TabController management) but no evidence of advanced process-death handling, state restoration, structured concurrency or background work tied to platform limits.

Evidence

ShopMate/lib/src/features/screens/sales/sell_screen.dart: TextEditingController lifecycle and dispose()

ShopMate/lib/src/features/screens/reports/reports_screen.dart: WidgetsBinding.instance.addPostFrameCallback and TabController init/dispose

ShopMate/lib/src/features/screens/sales/screen/all_sales_screen.dart: TabController lifecycle handling

Mobile UI/UX & Responsiveness

4/10

Smooth mobile experience

Strong, pragmatic UI work and responsive layouts for common phone sizes with dark-mode theming and polished widgets; not specialized for foldables, tablets, RTL or accessibility tuning.

Evidence

ShopMate/lib/src/features/screens/sales/sell_screen.dart: complex, theme-aware sales screen with AnimatedContainer, Dismissible and responsive paddings

ShopMate/lib/src/features/screens/sales/screen/all_sales_screen.dart: themed header, TabBar, dynamic lists and date picker theming

ShopMate/lib/src/features/screens/reports/reports_screen.dart: charts, tabbed UI and multiple adaptive stat cards

Performance & Battery

2/10

Speed and battery use

Some efficient widgets (ListView.builder, minimal rebuild patterns) but no measured performance work, profiling, baseline profiles or explicit battery-aware background handling.

Evidence

ShopMate/lib/src/features/screens/sales/screen/all_sales_screen.dart: uses ListView.builder for long lists

ShopMate/lib/src/features/screens/sales/sell_screen.dart: uses AnimatedContainer and limited animation timing control

Offline & Data Sync

2/10

Working offline and syncing

There is local persistence and provider usage suggested by imports and providers, but no visible offline sync engine, migrations, conflict resolution or queued idempotent retries in the analyzed UI code.

Evidence

ShopMate/lib/src/features/screens/sales/sell_screen.dart: checks stockProvider.getStockByName(...) before recording sales

ShopMate/lib/src/features/screens/reports/reports_screen.dart: reportProvider.setProviders(sales: salesProvider, stock: stockProvider) and reportProvider.loadReports() indicating local data aggregation

Device Integration

1/10

Using device features

No demonstrated device integration beyond UI labels (e.g., M-Pesa shown as a payment option); no permission flows, camera/location/biometric integrations or deep link handling evident.

Evidence

ShopMate/lib/src/features/screens/sales/sell_screen.dart: UI for PaymentMethod including 'M-Pesa' but no payment SDK integration or permission handling

Release & App Lifecycle

1/10

Building and publishing apps

Minimal release or lifecycle engineering artifacts in the analyzed human-authored files; pubspec shows common dev tools but there is no CI/CD, crash-reporting wiring or signing/release configuration in human files examined.

Evidence

ShopMate/pubspec.yaml: flutter_lints and flutter_launcher_icons listed (basic tooling)

Top-level project lacks human-authored CI or fastlane configuration in the HUMAN-AUTHORED files analyzed

Expertise

Flutter· Middle

Mobile UI/UX· Middle

Industries

Commerce· Middle

Financial Services· Middle

Technologies

Dart· since 2025 · Middle

Lottie

Flutter

Declarative UI

State Management

Material Design

Recommendations

Lead development of feature-rich Flutter screens and local app flows (inventory, sales, reports) for small-to-medium commerce apps.
Implement offline-first persistence and a simple sync layer (sqflite local store, migrations, queued idempotent uploads) to raise trust for real-world usage.
Add automated tests and CI (widget tests, integration tests, GitHub Actions or Codemagic) and wire basic crash reporting to improve reliability.
Expand device integration (payment gateway SDKs, permission flows, deep links) and add measurable performance profiling before/after optimizations.

Repositories

The developer's experience in this domain has been verified based on AI analysis of the following repositories:

ShopMate CreditPassport Flutter_basics

Job preferences

Work permits: Kenya
Citizenship: Kenya
Employment type: Full-Time

Open to AI agents: they read this profile and start a dialog through Alionalion.io/@derick-juma/mcp