Data Engineer (Oil&Gas)

Responsibilities

• Development of a platform for gas sales management and compliance management with the associated international standard;
• As part of the in-life management of existing data pipelines, you are responsible for maintaining all deliverables up to date, including the E2E “as-built” views of all data products in your scope;
• You and your team take an active part in the operational support process by providing assistance in case of incidents requiring a code-based fix or workaround;
• Create data pipelines;
• Transformations and ETL activities in Synapse;
• Data ingestion from various structured (SQL) and unstructured sources (XLS, CSV, Sharepoint files);
• ETL process orchestrations with Synapse and Azure Data Factory.

Qualifications

• Azure cloud services, including but not limited to Azure Synapse Analytics;
• Azure Data Factory, Azure Stream Analytics (real-time streaming & analytics);
• Azure Databricks with PySpark, Azure Blob Storage & Data Lake Storage, Azure Event Hubs, Azure Analysi;
• Services for data models, Azure Application Insights;
• Advanced knowledge in using Databricks;
• Azure Datalake.