Senior Data Engineer

Job Description:

- Proficiency in building highly scalable ETL and streaming-based data pipelines using Google Cloud Platform (GCP) services and products like Bigquery, Cloud Dataflow

- Proficiency in large scale data platforms and data processing systems such as Google BigQuery, Amazon Redshift, Azure DataLake

- Excellent Python, PySpark and SQL development and debugging skills, exposure to other BigData frameworks like Hadoop Hive would be added advantage

- Experience building systems to retrieve and aggregate data from event-driven messaging frameworks (e.g. RabbitMQ and Pub/Sub)

- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

- Work with data and analytics experts to strive for greater functionality in our data systems.

- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

- Keep our data separated and secure across national boundaries through multiple data centres and Azure/AWS regions.

Preferred Skills :

- Advanced working SQL knowledge and experience working with relational databases, query authoring, optimizing (SQL) as well as working familiarity with a variety of databases.

- Experience building and optimizing 'big data' data pipelines, architectures, and data sets.

- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

- Strong analytic skills related to working with unstructured datasets.

Secondary Skills :

- CloudBigTable, AI/Ml solutions, Compute Engine, Cloud Fusion

Company Profile :

ClicFlyer provides SAAS Business Intelligence and Analytics Solutions to Retail Chains and Brands across the Middle East leading Global FMCG Companies. From data integration and standardization to tailor-fit action plans, we take care of all the large and complex data to provide the retailer with customer Business Intelligence and Analytical centric insights. Our services are available in UAE, KSA, Bahrain, Qatar, Kuwait, Oman, Jordan, Egypt, S. Africa & Indonesia.

Skills

Data Engineering, Data Management, AWS, Python, SQL, ETL, Data Pipeline, Cloud, Google Cloud Platform, Big Data, Hadoop