History since 1997

Three sources, by age

Each stretch of time uses the best data published for it. Going back from now:

1. The last seven days: the real-time dispatch feed, every 5 minutes, with gaps filled from the daily dispatch files.
2. Back to the latest metered month (about five to six weeks): the EA’s daily dispatch files, the same dispatch cases, arriving 1–4 days late.
3. Every month the EA has published metering for, back to August 1997: metered data, what each station actually delivered to the grid, as the market settles it.

Prices are the EA’s final prices throughout the history, and HVDC is metered wherever published (about two weeks behind). The history job runs every six hours, adds months as the EA publishes them, and rebuilds every month whenever the plant register, the corrections or the metered plant table change.

A metered month

A month is built from the EA’s monthly metered files, half-hour by half-hour:

Generation

Generation_MD: each station’s output by connection point and generator code. Each code is mapped to a plant, fuel and island in facilities/metered_plants.csv (118 generator codes plus the old embedded meters, with date ranges and shares), and connection points the dispatch data never uses, many of them retired, get their island from the EA’s network supply points table. The build warns about any code it doesn’t know.

Embedded stations

Generation_MD covers stations on local networks only from January 2009. Before that, the site uses Embedded_generation under the old meter codes (Rotokawa RKA0111, Ngāwhā NGA0331, Kaimai KMI0331, Te Rapa TRC0331, Tararua TWF0331/2, Highbank HBK0661 and others). Only stations the site also shows later are included, so totals don’t jump in 2009.

Net meters

At some embedded points Generation_MD records output net of local load (Mangahao about 10% of its gross output, Cobb about 35%, Whareroa about 50%). Where the embedded meter at a listed point reads more, the difference is added to that point’s plant.

Shared meters

Some meters see two plants, split by their shares in the dispatch data: Deep Stream hydro and Mahinerangi wind (30/70, from June 2011); Te Ahi O Maui and Kawerau’s TOPP2 (35/65, from December 2025). Pātea’s three unit rows (from June 2026) each repeat the station total, so one is used.

Double readings

Trustpower’s embedded meters (Kaimai, Rotokawa, Tararua, Highbank, Paerau, Waipori, Teviot) read double until July 2005: a month whose peak is more than 1.4 times the plant’s nameplate is halved.

Gaps filled from dispatch

Where the metering lacks a plant, its dispatch figures are used: batteries (always, as dispatch nets their charging), solar farms not in the metering yet, and any plant whose metered energy is under half its dispatch energy in a month.

Demand, Tiwai, HVDC

From the metered grid flows; see demand and HVDC.

Embedded solar

The same model with each month’s installed capacity: none before the registry starts in August 2013, ERA5 reanalysis sunshine for 2013–15, the satellite from 2016. See embedded solar.

Plants that have closed are included: New Plymouth, Otahuhu A and B, Southdown, Te Awamutu, and Taranaki Combined Cycle (within Stratford).

Where the sources hand over

Over September 2020 to September 2026, where both exist, dispatch and metered figures agree closely (half-hourly national correlation 0.999 every month), but dispatch totals run 1–2% above metered ones every year: wind 3–4%, hydro 1–2%, geothermal, gas and coal within 2%. That is most likely because dispatch measures output at the generator and metering at the grid connection. The site uses each as published rather than scaling one to the other, so where the history switches from metered to dispatch data, about six weeks back, totals can step up slightly.

Plant by plant, the two can differ more. The Waikato stations are dispatched as one block, so their dispatch figures are the chain’s total split between stations, not each station’s output (correlation only 0.4–0.7 by half-hour); the metered figures are the real ones. Glenbrook’s metered output is about 1.45 times its dispatch, which sees only part of the plant.

Resolution

• The 30-day view shows half-hours.
• 1 year and All show days, or weeks, months and years. Each is the average over its half-hours, with peak demand and peak generation from each day’s highest half-hour.
• Energy figures add up the half-hours, so they are the same at any interval.

Checks

• Against MBIE’s annual net generation, which also counts industrial cogeneration and small embedded plants the EA’s metering doesn’t see: the total is 1.4–3.2% below MBIE in 2000–08 and 3.9–5.7% below in 2009–19; hydro is 1.3–2.9% below throughout; geothermal within ±5% from 2001.
• Emissions follow the national inventory year to year (see emissions).
• Metered demand matches the dispatch data’s island load within 1–2% where both exist.

Known gaps

• 1–9 September 2020: no Huntly. Generation_MD has no rows for the Rankines or unit 5 that month, and the dispatch files start on 10 September.
• Matahina from October 2021: no metered rows; the dispatch figures stand in.
• Before April 1999: Embedded_generation doesn’t yet have Rotokawa, Ngāwhā, Kaimai and Tararua, about 1–2% of generation in 1997–99. Geothermal in 1998–2000 is up to 15% below MBIE’s for that reason.
• Highbank before October 2001: no metered rows for it at all; it appears in Embedded_generation from then (25 MW).
• Mill Creek, April 2014 to August 2015: its first 17 months aren’t in Embedded_generation or Generation_MD (60 MW of wind, about 330 GWh). They are in the EA’s reconciliation data, and are due to be added at the next full rebuild.
• Grid imports before 2003: incomplete (see demand), so the stations’ own metered generation stands in.
• Owners: today’s owners apply to the whole history.
• Geothermal emissions before 2020 use 2020’s station factors, which probably understate them.

Code: ingest/history.py (the history job), ingest/metered.py (metered months). Evidence: analysis/metered_check.py, analysis/README.md.