2,432,254open jobs
150,075companies
296,122added this week
Browse all
Location
In office (India)
Seniority
Staff · 10+ years exp
Visa
Sponsors visas in United States; no register of this country names it

First seen by Alion on Nov 20, 2026. Egen scores B on the Alion truth index.

Overview
Company
Impact
Profile match
Egen is an American data, cloud and artificial intelligence consulting firm that designs and builds data platforms, cloud applications and AI solutions for enterprises, startups and public sector organisations. Its services cover data engineering on platforms such as Databricks and Google Cloud, application modernisation, cloud migration and generative AI implementation. The firm runs delivery teams in Chicago, San Francisco and Hyderabad, India. The company is headquartered in Naperville, Illinois, United States.

Job Overview:

We are looking for a skilled and motivated Lead Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for requirements gathering, designing, architecting the solution, developing, and maintaining robust and scalable ETL & ELT data pipelines. The role involves working with customers directly, gathering requirements, discovery phase, designing, architecting the solution, using various GCP services, implementing data transformations, data ingestion, data quality, and consistency across systems, and post-delivery support.

Experience Level:

10+ years of relevant IT experience

Key Responsibilities:

- Design, develop, test, and maintain scalable ETL data pipelines using Python.

- Architect the enterprise solutions with various technologies like Kafka, multi-cloud services, auto-scaling using GKE, Load balancers, APIGEE proxy API management, DBT, using LLMs as needed in the solution, redaction of sensitive information, DLP (Data Loss Prevention) etc.

- Work extensively on Google Cloud Platform (GCP) services such as Dataflow, Cloud Functions, BigQuery, Cloud Composer, Google Cloud Storage (GCS), IAM, and Cloud Run.

- Perform data ingestion from various sources and apply transformation and cleansing logic to ensure high-quality data delivery.

- Implement and enforce data quality checks, validation rules, and monitoring.

- Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver efficient data solutions.

- Manage version control using GitHub and participate in CI/CD pipeline deployments for data projects.

- Write complex SQL queries for data extraction and validation from relational databases such as SQL Server, Oracle, or PostgreSQL.

- Document pipeline designs, data flow diagrams, and operational support procedures.

Required Skills:

- 10+ years of hands-on experience in Python for backend or data engineering projects.

- Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.).

- Solid understanding of data pipeline architecture, data integration, and transformation techniques.

- Experience in working with version control systems like GitHub and knowledge of CI/CD practices.

- Experience in Apache Spark, Kafka, Redis, Fast APIs, Airflow, GCP Composer DAGs.

- Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.).

- Experience with PySpark is required.

- Experience in data migrations from on-premise data sources to Cloud platforms.

Good to Have (Optional Skills):

- Experience working with the Snowflake cloud data platform.

- Hands-on knowledge of Databricks for big data processing and analytics.

- Familiarity with Azure Data Factory (ADF) and other Azure data engineering tools.

- Experience with AWS services.

Additional Details:

- Excellent problem-solving and analytical skills.

- Strong communication skills and ability to collaborate in a team environment.

Education:

- Bachelor's degree in Computer Science, a related field, or equivalent experience.

Skills

Data Engineering, Python, Google Cloud Platform, BigQuery, Apache Spark, PySpark, Kafka, SQL, FastAPI, CI/CD, Apache Airflow

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,432,254 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
India
In office · Bengaluru
Databases
Databricks
Apply
In office · Full-Time · 7+ years exp · Bachelor's Degree · Vadodara
Python
SQL
PowerShell
Databases
PostgreSQL
Snowflake
MS SQL
DevOps
Azure
CI/CD
AWS
Amazon EC2
Amazon S3
IAM
Amazon CloudWatch
Apply
In office · Full-Time · 10+ years exp · Bachelor's Degree · Bengaluru
SQL
Databases
Oracle
DevOps
Azure
Apply
In office · 3+ years exp · Gurgaon · Bengaluru
Python
AI/ML
AI Agents
Machine Learning
Apply
Staff Data Scientist 3 hours ago
In office · Full-Time · 8+ years exp · Bachelor's Degree · Bengaluru
Python
AI/ML
Hadoop
Spark
MLFlow
SHAP
Triton Inference Server
Scikit-learn
TensorFlow
Pandas
NumPy
Keras
PyTorch
Amazon SageMaker
Machine Learning
DevOps
CI/CD
AWS
Kubernetes
Amazon EKS
AWS Lambda
Amazon EC2
Amazon S3
Amazon CloudWatch
Analytics
Seaborn
Matplotlib
Apply
Hybrid · 4+ years exp · Bachelor's Degree · India
Analytics
Microsoft Excel
Management
Xero
Apply
In office · Full-Time · 1+ year exp · Master's Degree · India
Apply
In office · Full-Time · Paris
C
Assembly
C
Embedded C
DevOps
RTOS
Apply
In office · Full-Time · Paris
Apply
In office · Full-Time · Paris
C
C
Embedded C
Management
Agile
Apply
See all jobs
This is one of many
2,432,254 more open roles from verified company boards, updated every day.