Flax / orbax β Checkpoint Metadata Integrity Gap
Summary
orbax.checkpoint.PyTreeCheckpointer.metadata() reads structure information
(shape/dtype) from _METADATA JSON. restore() reads actual tensor values
from OCDBT binary files. These are separate subsystems with no cross-validation.
A checkpoint directory can contain an unchanged _METADATA while the OCDBT
tensor data files contain different values. metadata() returns the original
structure; restore() loads the different values without an integrity warning.
Reproduction
pip install "jax[cpu]>=0.4.0" orbax-checkpoint flax
python3 create_flax_orbax.py artifacts/
# β creates baseline_ckpt, replacement_ckpt, tampered_ckpt
python3 inspect_metadata.py artifacts/
# β METADATA_CONTAINS_VALUE=False
# β METADATA_CHECKSUM_FIELD=False
# β INTEGRITY_GAP_CONFIRMED=True
python3 reproduce_restore.py artifacts/
# β RESTORED_BASELINE_VALUE=1.0
# β RESTORED_TAMPERED_VALUE=999.0
# β OUTPUT_FLIP_CONFIRMED=True
# β WARNING_EMITTED_METADATA_PATH=False
# β METADATA_INTEGRITY_GAP_CONFIRMED=True
Key Findings
| Check | Result |
|---|---|
metadata() contains tensor values |
False |
metadata() contains checksums |
False |
metadata() detects data replacement |
False |
restore() loads replacement values |
True |
| Integrity warning emitted on restore | False |
| OUTPUT_FLIP_CONFIRMED | True (1.0 β 999.0) |
Files
| File | Purpose |
|---|---|
create_flax_orbax.py |
Create baseline / replacement / tampered checkpoints |
inspect_metadata.py |
Demonstrate metadata() cannot detect data replacement |
reproduce_restore.py |
Confirm OUTPUT_FLIP (1.0β999.0), no integrity warning |
requirements.txt |
Python dependencies |
expected_output.txt |
Expected terminal output |
SHA256SUMS_T1.txt |
File integrity hashes |
logs/ |
Execution logs for all steps |
Environment
- jax 0.6.2
- flax 0.10.7
- orbax-checkpoint 0.11.37
- Python 3.10
Note on Warnings
The "Sharding info not provided" warning emitted during restore() is a
topology advisory that appears identically for both baseline and tampered
checkpoints. It is not related to data integrity and provides no signal
that tensor data has changed.
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support