Senior data Engineer

Job Description:

The ideal candidate is adept at using large data sets to find opportunities for product and process optimization and using models to test the effectiveness of different courses of action. Must have strong experience using a variety of data mining/data analysis methods using a variety of data tools building and implementing models using/creating algorithms and creating/running simulations. They must have a proven ability to drive business results with their data-based insights. They must be comfortable working with a wide range of stakeholders and functional teams. The right candidate will have a passion for discovering solutions hidden in large data sets and working with stakeholders to improve business outcomes.

Strong experience with various Data Management architectures like Data Warehouse Data Lake Data Hub and the supporting processes like Data Integration Governance Metadata Management

Strong ability to design build and manage data pipelines for data structures encompassing data transformation data models logics schemas metadata and workload management.

Strong experience in working with large heterogeneous datasets in building and optimizing data pipelines pipeline architectures and integrated datasets using traditional data integration technologies. These should include ETL/ELT data replication/CDC message-oriented data movement API design and access and upcoming data ingestion and integration technologies such as stream data integration

At least three years or more of work experience in data management disciplines including data integration modeling optimization and data quality and/or other areas directly relevant to data engineering responsibilities and tasks.

Collaborating with various teams and providing periodic updates through presentations and prototype demonstrations. The role will require working on multiple projects simultaneously. This position will also be involved in the formulation of key business requirements to be solved rationalizing the various analytical approaches to solve those problems.

Maintains state-of-the-art knowledge and contributes to technical discussions and reviews as an expert in related areas of responsibility

Communicates ideas plans and results effectively via oral and written reports - Works effectively with peers management operations groups and outside organizations

Participate in technical forums and other appropriate events and conferences

This position is involved in helping to develop analyze and draw conclusions and presenting the results back to business users. Depending on the candidates qualification this position can be either a Data Engineer or a Data Scientist.

Qualifications:

Strong ability to design build and manage data pipelines for data structures encompassing data transformation data models schemas metadata and workload management.

The ability to work with both IT and business in integrating analytics and data science output into business processes and workflows.

Strong experience with popular database programming languages including SQL NO SQL

Strong experience in working with large heterogeneous datasets in building and optimizing data pipelines pipeline architectures and integrated datasets using traditional data integration technologies. These should include ETL/ELT data replication/CDC message-oriented data movement API design and access and upcoming data ingestion and integration technologies such as stream data integration CEP and data virtualization.

Strong experience in working with SQL on Microsoft SQL Server.

Strong experience in working with and optimizing existing ETL processes and data integration and data preparation flows and helping to move them in production.

Basic experience working with Power BI for semantic-layer-based data discovery.

Strong experience of using Alteryx as ETL tool and Qlik as BI tool.

Other Skills and Abilities:

Strong ability to work in a fast-paced environment

Strong ability to work both autonomously and in a team

Strong ability to communicate complex quantitative analysis in a clear precise and actionable manner to both technical and non-technical audiences

Excellent time-management multi-tasking and communication skills. Capable of juggling multiple projects and related complexities at the same time

Good interpersonal and communication skills

Knowledge in Hadoop ecosystems will be a plus

Have a good understanding of latest technology trends