Data Specialist

We need someone who understands data deeply and uses Python to wrangle it — not a platform engineer, not a pure pipeline builder, but a data specialist who's comfortable with research, investigation, and the unglamorous work of making messy energy market data actually usable.

You'll spend significant time on tasks like: mapping BM units to power plants and fuel types, reconciling legacy data formats with current ones, ensuring consistency between different Elexon message types, and cleaning time-series data (outliers, gaps, overlaps). Some of this requires genuine investigation — cross-referencing sources, making judgment calls, documenting edge cases. There's no API that solves these problems for you.

Python is your primary tool (Pandas, Numpy, standard libraries) to minimise manual effort, but you should be comfortable that some detective work is unavoidable. If you find satisfaction in truly understanding a dataset's structure and quirks — rather than just piping data through and hoping for the best — this role is for you.

Data Mapping and Research

• Map BM units from Elexon to their corresponding power plants, substations, and fuel types — combining API data, public registers, and manual research

• Map substations to ETYS zones and grid supply points

• Build and maintain reference/master datasets that link identifiers across disparate sources (Elexon, National Grid ESO, TEC register, etc.)

• Document mappings, assumptions, and known limitations clearly for downstream users

Data Reconciliation and Consistency

• Reconcile legacy data formats with current formats (e.g., historical operational data stored in different schemas or granularities)

• Ensure consistency between different Elexon message types — understand the market data structure well enough to know why BOALF, BOD, and DISBSAD might not perfectly align and how to handle it

• Investigate discrepancies between data sources and determine authoritative values

Data Cleaning and Quality

• Clean time-series data: detect outliers (price spikes, meter errors), fill gaps appropriately, resolve overlapping or duplicate timestamps

• Develop reusable Python-based cleaning routines that can be applied across datasets

• Understand why data quality issues occur (settlement reruns, late submissions, format changes) not just patch them

Pipeline Development (Supporting the Above)

• Write and maintain Python data grabbers for energy market APIs

• Build dbt models to transform raw data into clean, analysis-ready datasets

• Orchestrate workflows via GitHub Actions