Data Engineer

Strategic Staffing Solutions is currently looking for a Data engineer, for a W-2 contract opportunity.

Data Engineer

Location:Charlotte, NC,28206

Type: W-2 Contract (NO C2C)

Contract: 12 months

Schedule: Hybrid

This is a W-2 contract opportunity, and candidates must be able to work on our W-2 only. NO C2C or 1099.

Position Overview

Required Technical Skills

• Python: Advanced proficiency in Python 3.8+ for data processing, scripting, and automation
• PySpark: Extensive experience with PySpark for large-scale data processing and transformations on EMR clusters
• AWS Lambda: Hands-on experience building serverless functions for data processing and orchestration
• AWS Glue: Expertise in AWS Glue for ETL jobs, Data Catalog management.
• Amazon Athena: Experience with Athena for SQL-based data analysis and querying data lakes
• Amazon EMR: Strong experience with EMR for running Spark jobs, EMR Notebooks, and cluster management
• Amazon Redshift: Knowledge of Redshift for data warehousing, query optimization, and integration with data lakes
• Kafka: Experience with Confluent Kafka for real-time data streaming, including Schema Registry, source/sink connectors, and producers/consumers
• Amazon S3: Deep understanding of S3 for data lake architecture, bucket policies, and data organization (raw, curated layers)
• CloudWatch: Proficiency in CloudWatch for logging, monitoring, and alerting on data pipelines
• Terraform: Experience with Infrastructure as Code using Terraform for provisioning AWS resources

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines using AWS Glue, EMR (PySpark), and Lambda functions
• Implement real-time data streaming solutions using Confluent Kafka with source and sink connectors
• Develop and optimize PySpark jobs on EMR clusters for large-scale data transformations
• Build serverless data processing workflows using AWS Lambda
• Design and optimize queries in Amazon Athena for analytical workloads
• Manage data warehouse solutions in Amazon Redshift including data modeling and performance tuning
• Implement monitoring and alerting using CloudWatch for data pipeline health and performance
• Provision and manage AWS infrastructure using Terraform
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
• Implement data governance and security best practices using Lake Formation, IAM, and KMS

Additional Technical Experience

• Knowledge of AWS EventBridge and Step Functions for workflow orchestration
• Familiarity with EMR runtime roles and execution roles for secure data access
• Experience with DynamoDB for NoSQL data storage
• Knowledge of API Gateway for building data APIs

Preferred Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field
• Strong understanding of distributed computing and big data processing frameworks
• Experience with Agile development methodologies
• Excellent problem-solving and analytical skills
• Strong communication and collaboration abilities

Development and Data Tools

• EMR Notebooks or Jupyter for interactive development
• VS Code or similar IDE for Python development
• Git for version control
• AWS CLI and boto3 for AWS service interaction
• Terraform for infrastructure provisioning
• DBeaver, Data Studio for database management

“ Beware of scams. S3 never asks for money during its onboarding process.”