# Data vintages — S232 derivatives sizing (JCC-3892, FR 2026-15961)

Three import-value vintages back `article_sizing.csv` and `article_origins.csv`, all
USITC HS8 data, all structured-uncertainty status `answered`.

**Re-queue correction (2026-08-09).** The first attempt at this stage found no
`DATAWEB_API_TOKEN` and fell back to UN Comtrade's HS6 preview API, tagging 2025 and
2026 `proxy_used`. That fallback was unnecessary: 29 monthly USITC "M + TAU" workbooks
covering exactly this window were already on disk at
`jf-thought/sgept-analytics/data-queries/251212-us-effective-tariff-rates/data/`
(`USITC M + TAU YYMM.xlsx`, YYMM = 2401…2412, 2501…2512, 2601…2605). This run reads
those files directly. The Comtrade-based outputs from the first attempt are archived at
`results/archive/260809-s232-comtrade-proxy/` for comparison; the two Comtrade CSVs
moved to `data/archive/`.

**Second re-queue correction (2026-08-09, trade-flow basis).** The 2024 baseline and
the 2025/2026 workbook extracts run on two different USITC trade-flow concepts (see
"2024 reconciliation" below); an earlier fix diffed them directly and overstated one
article's 2025 growth by 55 percentage points. See "Trade-flow basis for change
figures" below for the corrected `article_sizing.csv` column layout.

## Source

- **File pattern:** `USITC M + TAU YYMM.xlsx`, one workbook per month.
- **Sheet:** `Customs Value` (the matching `Calculated Duties` sheet carries duty
  figures, not used by this pipeline).
- **Query Parameters** (verified identical across every workbook checked): Trade Flow
  = *Imports For Consumption*; Classification System = HTS Items; Data To Report =
  *Customs Value, Calculated Duties*; Commodity Aggregation Level = 8 (HS8); Country
  Aggregation = Break Out Countries.
- **Two export layouts** across the 29 files (confirmed by reading each file's header):
  - *Legacy* (2401–2412, 2501–2509 in this set): header on row 1; columns `Data Type,
    Country, Year, Month, HTS Number, Program, Customs Value`.
  - *Preamble* (2510–2512, 2601–2605 in this set): two title rows above the header, so
    the header is on row 3; columns `Data Type, Country, Year, HTS Number, Program,
    <MonthName>` (no `Month` column — the value column is named for the month itself).
  `code/s232_derivatives/build_usitc_vintages.R` detects the layout per file (presence
  of a `Month` column) and normalises both to the same shape before aggregating.
- **HTS Number:** read as an 8-digit character string by `readxl` in every file
  checked (leading zeros preserved); the script defensively reconstructs an 8-digit
  code from a numeric column if a sheet is ever parsed that way, and asserts
  `nchar == 8` on every row it keeps.
- **Extraction date:** 2026-08-09, via `Rscript code/s232_derivatives/build_usitc_vintages.R`.

## Target scope

19 distinct HS8 lines behind the 14 proposed derivative articles, read from
`data/fr_2026_15961/annex_articles.csv` (the official-source parser output — see
`documentation/260809-s232-derivatives-fr-2026-15961/SOURCE.md`), never re-derived by
this script.

## The three vintages

| Vintage | Months | File | Role |
|---|---|---|---|
| 2024 full year | 2401–2412 | `data/usitc_s232_derivatives_2024_full_rebuild.csv` | **Reconciliation only** — see below. Not the pipeline's 2024 baseline. |
| 2025 full year | 2501–2512 | `data/usitc_s232_derivatives_2025_full.csv` | Primary 2025 vintage in `article_sizing.csv` / `article_origins.csv`. |
| 2026 YTD Jan–May | 2601–2605 | `data/usitc_s232_derivatives_2026_ytd_jan-may.csv` | Primary 2026 vintage. Every 2026 figure is labelled `2026-YTD-jan-may` and is **never annualised** — the workbook set simply stops at 2605 (May 2026 is the latest month with a monthly extract on disk). |

**The pipeline's 2024 baseline stays `data/usitc_us_imports_2024.csv`** — the existing,
unchanged, repo-wide canonical file that every other S232 script here already
cross-checks against (`check_origin_shares.R`'s spot-check, `METHODOLOGY.md` §1.1,
`code/engine/us_tariff_calculation.R`). `build_article_sizing.R` reads 2024 from that
file, not from this stage's rebuild. The rebuild exists solely so this stage can
independently confirm — from the raw monthly workbooks, the same way 2025/2026 are
built — that the canonical file is trustworthy before the two new vintages are trusted.

## 2024 reconciliation

`results/s232_derivatives/vintage_2024_reconciliation.csv` compares the independent
2024 rebuild (12 months, summed by HS8 across all countries) against the same 19 HS8
lines pulled from `data/usitc_us_imports_2024.csv`.

**16 of 19 HS8 lines reconcile within ±3.5%** — consistent with ordinary
month-to-month rounding and minor reporting-lag noise, not a methodology problem.

**3 of 19 diverge materially:**

| HS8 | Article | Canonical (existing file) | Rebuild (monthly workbooks) | Diff |
|---|---|---:|---:|---:|
| 29012200 | Filled steel containers — propene/propylene | $66.0M | $359.5M | +444.7% |
| 27111200 | Filled steel containers — propane | $1,894.9M | $2,535.9M | +33.8% |
| 84261200 | Mobile lifting frames on tires and straddle carriers | $134.4M | $151.2M | +12.5% |

**Why:** the workbooks' Query Parameters sheet states Trade Flow = *Imports For
Consumption*. `data/usitc_us_imports_2024.csv`'s own `data_type` field reads *"General
Customs Value"*, which (going by the label) is USITC's *General Imports* trade flow at
Customs Value — a different Census trade-flow concept. General Imports counts
merchandise on arrival in US customs territory (including goods entering bonded
warehouses / foreign-trade zones); Imports for Consumption counts merchandise that has
formally cleared customs for consumption, which is not a fixed-lag subset of General
Imports in any given calendar year because it also includes withdrawals from earlier
inventory. For ordinary manufactured goods the two series track closely (matching what
16 of 19 lines here show). Bulk liquid petrochemicals — propane and propylene, both
commonly held in bonded storage or FTZs at import terminals before formal entry — are
exactly the product class where the two trade-flow bases are known to diverge, which
matches the two largest gaps in this table. HS8 84261200 (mobile lifting frames) is a
capital-equipment line where temporary FTZ dwell before entry is also plausible,
though this stage did not independently verify the mechanism for that line beyond
confirming the trade-flow-basis explanation is directionally consistent.

**Consequence:** this is a genuine trade-flow-definition difference between two
legitimate USITC statistics, not a data error in either file. No figure in this
pipeline is wrong; two USITC series answer slightly different questions for these
three codes, the same way the CH/China-Brazil MCP-vs-blog reconciliation
(`documentation/advisories/260609-brazil-s301-mcp-activation/MCP_vs_blog_reconciliation.md`)
found two engines answering different questions about Chapter 98 inclusion. The 2024
baseline this pipeline actually uses for its published **level**
(`usitc_us_imports_2024.csv`, General Imports basis) is unaffected by this finding — it
is unchanged and remains the anchor for repo-wide comparability. But the reconciliation
means `imports_2024_usd` (General Imports) and `imports_2025_usd` /
`imports_2026_ytd_usd` (Imports For Consumption) are two different trade-flow bases and
must never be diffed against each other to produce a change figure — see "Trade-flow
basis for change figures" below for how this pipeline now handles that.

## Trade-flow basis for change figures (second re-queue correction)

**A first version of this fix computed `pct_change_2025_vs_2024` as
`imports_2025_usd / imports_2024_usd − 1` — but `imports_2024_usd` is General Imports
while `imports_2025_usd` is Imports For Consumption. Diffing the two produced a change
figure computed across two trade-flow bases, caveated only on articles 9 and 14.**
Measured against a like-for-like (same-basis) comparison, the mixed-basis figure overstated
article 14 (filled steel containers, the largest article in the set at a $1.96bn 2024
level) by 55 percentage points — +70.6% mixed-basis vs **+15.6%** same-basis — and
moved article 9 (mobile lifting frames) from -16.9% to **-26.2%**, and article 13
(other trailers and semi-trailers) from +24.4% to **+28.1%**. The other eleven articles
moved by less than 1.5 points, because their 2024 canonical and rebuild totals are
close (the ±3.5%-or-better group above).

**Fix:** `article_sizing.csv` now carries the level and the change figure as two
separate, explicitly basis-labelled column groups, and computes every change figure on
a single Imports For Consumption basis:

| Column | Basis | Role |
|---|---|---|
| `imports_2024_usd` | General Imports (Customs Value) — `usitc_us_imports_2024.csv`, canonical, unchanged | The pipeline's published 2024 **level**. Repo-wide comparability; never used as a change-figure denominator. |
| `imports_2025_usd`, `imports_2026_ytd_usd` | Imports For Consumption — USITC monthly workbooks | Published 2025 / 2026-YTD levels. |
| `imports_2024_ifc_usd` | Imports For Consumption — this stage's own 2024 rebuild, `data/usitc_s232_derivatives_2024_full_rebuild.csv` | **Reconciliation-only.** Never published as a level; exists solely as the same-basis 2024 term for the change figure below. |
| `pct_change_2025_vs_2024_ifc` | Imports For Consumption on both terms (`imports_2025_usd` over `imports_2024_ifc_usd`) | The pipeline's sole sanctioned 2025-vs-2024 change figure. |
| `pct_change_basis`, `basis_note` | — | Per-row text stating which basis each figure runs on and warning against cross-basis diffs. |

The old `pct_change_2025_vs_2024` and per-article `trend_caveat` columns are removed —
the trade-flow-basis problem was not specific to articles 9 and 14 (they were simply
the two articles large enough for it to move the headline number by a visible margin;
the fix now applies uniformly to all 14). **The S232 post must quote
`pct_change_2025_vs_2024_ifc`, never a figure computed by hand from `imports_2024_usd`
and `imports_2025_usd`.** Gate G4 (`gate/JCC-3892/g4-trade-flow-basis-change-figures`,
journaled to `plans/decision-journal.jsonl`) records this as a durable data-contract
decision.

## 2026 YTD cutoff

Gate G2 (`gate/JCC-3892/g2-ytd-cutoff`, journaled to `plans/decision-journal.jsonl`):
2026 YTD = **Jan–May 2026 cumulative (5 months)**, labelled `2026-YTD-jan-may`
throughout, never annualised. The cutoff is mechanical here — the monthly workbook set
on disk stops at `2605.xlsx` — not a reporting-lag judgement call as the first
attempt's Comtrade-based cutoff reasoning had it. The underlying decision (May) is
unchanged from the first attempt; only the reason it is May has changed.

## What this means for the S232 post

- All vintages and terms (including the reconciliation-only 2024 IFC rebuild) are
  HS8-precise, directly-observed USITC data — no `proxy_used` caveat is needed
  anywhere in this pipeline's output.
- Quote `imports_2024_usd`, `imports_2025_usd`, `imports_2026_ytd_usd` as levels only.
  Quote `pct_change_2025_vs_2024_ifc` as the only sanctioned 2025-vs-2024 change
  figure — it is already computed on a single Imports For Consumption basis for all
  14 articles, so no per-article caveat is needed on it.
- No 2026-YTD-vs-2024 change figure is computed anywhere in this pipeline (2026 YTD is
  5 months; diffing it against a 12-month 2024 level, on any basis, would not be a
  like-for-like comparison — see the annualisation ban in `CLAUDE.md`/the brief). If a
  2026 YTD change figure is wanted later, it needs its own Jan–May 2024 term, not the
  full-year terms in this table.

## Cross-vintage table

| Vintage | Label | Source | Basis | Role | Precision | Status |
|---|---|---|---|---|---|---|
| 2024 | `2024-full-year` | `data/usitc_us_imports_2024.csv` (existing, canonical) | General Imports (Customs Value) | Published level | HS8 | `answered` |
| 2025 | `2025-full-year` | USITC monthly workbooks, `data/usitc_s232_derivatives_2025_full.csv` | Imports For Consumption | Published level | HS8 | `answered` |
| 2026 YTD | `2026-YTD-jan-may` | USITC monthly workbooks, `data/usitc_s232_derivatives_2026_ytd_jan-may.csv` | Imports For Consumption | Published level | HS8 | `answered` |
| 2024 IFC rebuild | `2024-full-year-ifc-rebuild` | USITC monthly workbooks, `data/usitc_s232_derivatives_2024_full_rebuild.csv` | Imports For Consumption | Reconciliation only — 2024 term of `pct_change_2025_vs_2024_ifc`, never a published level | HS8 | `answered` |
