Data Engineer

Key Responsibilities Data Engineering & Pipelines Build and enhance data pipelines and automation workflows on Azure for large-scale data ingestion and processing Maintain and optimise existing toolkits, databases, dashboards, and API-based solutions Develop end-to-end automated data solutions across commodities (Gas/LNG, Power, Weather) Design and implement web scraping pipelines (e.g. US LNG data) Analytics & Real-Time Systems Support real-time and batch analytics use cases in a dynamic trading environment Enable and support analytics dashboards and GUI tools with reliable backend infrastructure Develop real-time flow tracking solutions at critical market points Data Modelling & Architecture Design and manage Snowflake data models, ensuring performance and scalability Integrate external data sources including ENTSOE, EPEX, KPLER, MetDesk, WoodMac, and others Collaborate with analytics teams to structure data for models and advanced analytics use cases Build and maintain data platforms supporting ML models such as demand forecasting Key Deliverables LNG Sendout Optimisation Model Ship Tracking & Flow Monitoring Tools Prompt Price & Forward Curve Bootstrapping Web scraping pipelines for US LNG and other market data sources Real-time flow tracking infrastructure at critical market points Data platforms supporting ML and demand forecasting models Required Skills & Experience Technical Strong proficiency in Python for data engineering and pipeline development Hands-on experience with Azure data services (Data Factory, Databricks, Blob Storage, etc.) Experience designing and optimising Snowflake data models Proven track record building and maintaining data pipelines at scale Experience integrating third-party market data APIs and external data feeds Familiarity with SQL and data warehousing concepts Domain Experience or strong interest in energy, commodities, or financial markets data Understanding of real-time and batch data processing patterns in a trading context Familiarity with market data providers such as ENTSOE, EPEX, KPLER, or similar Nice to Have Experience with web scraping frameworks (e.g. Scrapy, BeautifulSoup, Playwright) Knowledge of LNG, Gas, or Power market fundamentals Experience supporting ML model pipelines or demand forecasting workflows Exposure to GUI/dashboard tools such as Grafana, Power BI, or Streamlit Knowledge of ship tracking data or AIS feeds