Correct TabSketchFM checkpoint provenance and re-host decision

[weipang142857]

Problem

docs/CHECKPOINT_LICENSES.md recorded the TabSketchFM checkpoint as an IBM/LakeBench artifact and directed users to Zenodo (10.5281/zenodo.8014642) to obtain it. That was wrong on two counts:

1. Upstream released no checkpoint at all. The TabSketchFM authors published the code, but no pretrained weights and no pretraining corpus.
2. The weights TRL-Bench uses were trained by us — from scratch, with the upstream pretrain.py, initialized from bert-base-uncased, over tables fetched from the source URLs upstream publishes in pretraining_tables.txt.

Consequence: anyone following the documentation to reproduce the paper's TabSketchFM results hit a dead end.

Evidence

The checkpoint's own metadata records pytorch-lightning 2.5.6 — a release postdating the upstream paper (arXiv:2407.01619, July 2024) — so it cannot be an upstream artifact.

The filename epoch=10-step=27786.ckpt is PyTorch Lightning's default ModelCheckpoint auto-naming and carries no provenance; the original entry appears to have inferred IBM origin from it. The file is also a weights-only save (state_dict only, no hyper_parameters or callbacks), which is why it embeds no training paths that would have settled the question directly.

Change

Four corrections in docs/CHECKPOINT_LICENSES.md:

• Summary table row — re-host NO (no derivatives) → weights are ours, mirrored
• Main TabSketchFM section — records that no upstream checkpoint exists, the provenance of ours, the version evidence, and the re-host rationale
• "Checkpoints documented upstream-only" list — TabSketchFM removed from it
• Fetch-path table — Zenodo → HF mirror

Re-host rationale, in short: the ND clause governs redistribution of code, and these weights are our own training output rather than a copy of an IBM-distributed artifact. Publishing is additionally necessary for reproducibility, since no upstream checkpoint exists.

Companion change (already live, outside this repo)

epoch=10-step=27786.ckpt is now mirrored at logo-lab/trl-bench-ckpts/tabsketchfm/ with per-model LICENSE and NOTICE. Remote SHA256 verified identical to local (26f2107d…f9f211d).

Weights only — no upstream source code is redistributed. The NOTICE states prominently that this is not an official IBM release and directs users to IBM/tabsketchfm for the code under the upstream license. The repo README was updated to match, including scoping its previous blanket "byte-identical to the upstream releases" claim to the checkpoints where it actually holds.

scripts/checksums.sha256 already carried the correct hash for this file, so post-download verification works unchanged.

Not included (deliberate — follow-up needed)

Two files still direct users to Zenodo and should be fixed separately:

• scripts/download_checkpoints.sh:85-89 — prints the placeholder and skips; TabSketchFM will not auto-fetch from the new mirror
• src/trl_bench/registry.py:964-968 — comment still reads "Cannot be mirrored (ND clause); users fetch from the IBM/LakeBench Zenodo record"
• scripts/checksums.sha256 — hash correct; only its (CC BY-NC-ND 4.0, upstream-only) comment label is stale

🤖 Generated with Claude Code

[Copilot]

🟡 Not ready to approve

A few documentation updates are confusing/inconsistent (upstream-only list) and include overly definitive license interpretation language that should be made more cautious/precise.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Updates checkpoint licensing/provenance documentation to correct TabSketchFM’s source attribution and to document the decision to mirror TRL-Bench-trained weights (since no upstream checkpoint exists), improving reproducibility guidance.

Changes:

• Corrects TabSketchFM provenance (no upstream checkpoint; TRL-Bench-trained weights) and updates the mirror decision accordingly.
• Removes TabSketchFM from the “upstream-only” checkpoint category and updates the fetch-path table to reference the HF mirror.
• Expands TabSketchFM section with provenance evidence and updated attribution guidance.

File summaries

File	Description
docs/CHECKPOINT_LICENSES.md	Corrects TabSketchFM checkpoint provenance/mirroring policy and updates user-facing fetch instructions accordingly.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 4
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.