Data Engineer

Skillset : Data Architect - Data Engineering, Distributed Systems, GCP Cloud Architecture

Experience Level : 8+ Years

Required Skills :

- Strong experience in designing and architecting large-scale data platforms andistributed data processing systems.

- Strong programming experience in Python, with hands-on experience in developing data processing and ETL/ELT pipelines.

- Hands-on experience with Apache Spark/PySpark, Kafka, and distributed data processing.

- Strong understanding of data architecture, data modeling, data integration, ETL/ELT pipelines, and data warehousing.

- Strong knowledge of SQL and NoSQL databases and experience in designing scalable data solutions.

- Strong hands-on experience with GCP cloud services and architecture, including BigQuery, GCS, Pub/Sub, Dataflow, Dataproc, Cloud Run, and Workflows.

- Strong understanding of distributed systems, event-driven architecture, scalability, high availability, and fault-tolerant system design.

- Experience with Docker, Kubernetes, and CI/CD practices.

- Experience in performance tuning and optimization of data pipelines and cloud-based data platforms.

- Strong understanding of cloud security, data governance, access management, and best practices on GCP.

- Excellent problem-solving, stakeholder management, technical leadership, and architecture decision-making skills.

Good-to-Have Skills :

- Experience with Apache Flink, Airflow, Databricks, or Snowflake.

- Experience with Terraform or other Infrastructure as Code (IaC) tools.

- Exposure to AI/ML data pipelines, GenAI, or Agentic AI solutions.

- Experience with observability and monitoring tools such as Cloud Monitoring, Prometheus, Grafana, ELK, or Splunk.

- GCP Professional Cloud Architect or Professional Data Engineer certification.

- Experience in mentoring teams and driving architecture standards, best practices, and governance.

Skills

Data Architect, Python, Apache Spark, Kafka, ETL, SQL, NoSQL, Data Engineering, Google Cloud Platform, Snowflake DB, Databricks