Skip to main content

Reproducible source archives

OpenDAL source release archives must reproduce byte for byte from the same Git revision and locked packaging dependencies. This contract covers the .tar.gz files generated by odev release, including every binding and integration source package. It does not claim reproducibility for Maven artifacts, wheels, native binaries, or OpenPGP signature files.

Archive inputs​

Build from a clean checkout of the exact candidate commit, with Git line-ending conversion disabled (core.autocrlf=false) and native symlink support. Run the packager with cargo run --locked --manifest-path dev/Cargo.toml -- release --unsigned. The packager reads tracked working-tree contents and the Git index's file modes; local source edits therefore change the release bytes. It does not regenerate source files as part of packaging.

The source package inventory comes from dev/src/release/package.rs. Each archive includes the package, its declared source dependencies, and the root LICENSE and NOTICE files. Paths are sorted. Tar entries use timestamp zero, numeric owner and group zero, empty owner/group names, and Git's regular/executable file modes. Symlinks retain their link target and are not followed. Gzip includes neither a filename nor a timestamp. The checked-in dev/Cargo.lock fixes the compression implementation. Changing those dependencies can change archive bytes and requires repeating the reproduction checks.

Reproduce the complete package set​

From a trusted checkout containing the verification script:

python3 scripts/verify_source_reproducibility.py \
  --revision <candidate-commit> \
  --output /tmp/opendal-reproduction

The output directory must not already exist. The script creates two independent checkouts, changes their file timestamps, permissions and process timezones, and builds every source archive in both. It records the commit, packaging patch digest, host and compiler information, all archive hashes, and build logs. A mismatch or missing package fails the command. The first set of archives is retained alongside the report. Temporary checkouts are removed automatically.

For local packaging-tool development, --working-tree applies tracked changes to both checkouts and retains the exact input patch in the output. Such a run is useful regression evidence, but release verification must use the committed candidate without this option.

To validate a CI candidate on trusted hardware, download the complete voted-on .tar.gz set to a directory and add:

python3 scripts/verify_source_reproducibility.py \
  --revision <candidate-commit> \
  --compare /path/to/downloaded-candidate \
  --output /tmp/opendal-candidate-reproduction

This compares the full filename set and every archive's SHA-512 against the local rebuilds. Verify the downloaded signatures separately and include both results in the release review. Reproducibility does not replace license review or testing.

odev CI performs the same check on Linux and macOS and compares their manifests. A CI-to-CI comparison is regression coverage; it does not replace the required independent reproduction on trusted hardware before publication.

See PR #8232 for the recorded reproduction evidence and the trusted publishing procedure for source-only signing and staging.