← Reproduction studies

Can a published climate normal be recomputed from the data it derives from?

Recomputing NOAA's 1991–2020 U.S. Climate Normals from the GHCN-Daily observations they are built on, across 12 stations, as a Facetwork FFL workflow. September 2026.

1. Purpose of the analysis

A climate normal is the reference against which "hotter than average" is measured — in agriculture, energy demand forecasting, insurance pricing and building codes. NOAA publishes the normals and the daily observations they derive from, so unlike most published results this one can be checked rather than trusted.

The question is not "are the published normals correct". It is narrower and more useful: can an independent party, starting from the same archived observations, arrive at the same numbers? Where the answer is no, the gap measures how much of the published product depends on decisions that are documented in prose but not reproducible from the archive.

2. What the original work does

NOAA NCEI's U.S. Climate Normals are not an average of the daily archive. Their published Normals Calculation Methodology (2020) describes a substantially longer pipeline, and the two steps that matter here are both corrections applied before any averaging:

NCEI even give a worked example of the homogenisation adjustment, at exactly the magnitude this study measures:

What we do differently — deliberately

So the difference we measure is not an error on either side — it is the size of NOAA's corrections. That reframes the whole study: it is not "can the normals be reproduced?" but "how much of the published normal comes from the raw record, and how much from the adjustments applied to it?" Those adjustments are station-specific by construction, which is precisely what the twelve-station spread shows.

Two things we assumed and the documentation corrects. The comp_flag value S does not mean 30 complete years — under the 2020 WMO guidelines it means at least 24 years (80%). And TAVG is not defined as (TMAX+TMIN)/2: NCEI note the average temperature normal "may not be identical, but within rounding error, of the sum of the corresponding maximum and minimum temperature normals". Our TAVG column uses the conventional midpoint, so a small part of its residual is definitional. Full field layout: the by-station readme.

3. What our FFL workflow found

Each station is fetched, recomputed (mean of daily values per calendar month, tenths °C → °F), and compared against the published normals. 12 stations, 61 tasks, distributed across a four-host fleet. 432 values in total (12 months × TMAX/TMIN/TAVG × 12 stations).

"Bias" is the mean signed residual (ours − NOAA) where an exact sign test finds the residuals systematically one-sided (p < 0.01 over 12 months); none means consistent with noise. The sign test is threshold-independent, so unlike the error magnitudes it does not move when the tolerance changes.

Los Angeles reproduces to 0.040 °F median; Chicago is off by 0.875. That is a twentyfold spread across stations processed from the same archive by the same code. Overall:

4. How our workflow compares to the published work

The algorithmic point: a tolerance cannot see a bias

At New York every one of 36 values sits within a generous tolerance and every one falls on the same side per column — TMAX high 12/12, TMIN low 12/12.

verdict : agree at level `values`,  differing: 0
TMAX residuals SYSTEMATIC: 12+/0- of 12, mean +0.2425  (p=0.00049)
TMIN residuals SYSTEMATIC: 0+/12- of 12, mean -0.5108  (p=0.00049)

Agrees and differs at once. "36 small random errors" and "36 errors in the same direction" are completely different claims about a reproduction, and only the second says the method differs. A tolerance is a per-value predicate; it cannot express that. So the comparison runs an exact sign test over the signed residuals — Binomial(n, ½) under the null, no distributional assumption — and reports the structure beside the verdict. Being threshold-free, it is the part of this study that does not move when the tolerance does.

What the verdict levels mean

The comparison returns the highest rung reached on a ladder, so the level names what did match and thereby locates the first thing that did not:

keys is the informative rung and the easiest to misread. It says both sides looked at exactly the same population and computed different answers — nothing lost in transfer, no filter diverged, no schema drifted. That is a finding about method. A verdict of cardinality or lower usually means something broke before the science started.

The key columns define what "the same record" means, which is a substantive claim rather than configuration. Here the key is (month): twelve records, all twelve present on both sides, so every disagreement is necessarily at keys — the months are the same, the temperatures are not. In the companion GIAB study the key is (CHROM, POS, REF, ALT); without it records would match by position in the file, and one inserted row would misalign everything after it.

5. Reproducing this

fw ffl run --primary examples/repro-normals/noaa_normals_repro.ffl \
           --library facetwork/ffl/file.ffl \
           --workflow repro.normals.ReproduceManyStations

The recomputation is an FFL script block bound to fw.file.FileOps — no handler and no domain package. The ~18 MB daily file is read inside the script rather than passed through a step payload, which would exceed the 16 MB document limit. foreach … limit 4 bounds how many stations are in flight so the run does not pull twelve large files from NOAA at once.

6. Limitations, and a correction

• Our method is deliberately naive, and the published value is the better one. A plain monthly mean omits the TOB adjustment and the homogenisation described in §2. Where the two differ, NOAA's figure is the more considered: it corrects for observation time, station moves and instrument changes that the raw archive still contains. This study measures the size of those corrections; it does not evaluate their correctness.
• Twelve stations, all US first-order airport/city sites. The pattern may not extend to cooperative-observer stations, which have different reporting practices and more missing data.
• TAVG is computed as (TMAX+TMIN)/2, the conventional definition but not necessarily how the published TAVG normal is derived.
• The sign test detects that residuals are one-sided. It does not attribute a cause; the station-specific explanations offered above are consistent with the data, not demonstrated by it.