Performance

Rivet’s capture path is the only part of the tool whose cost scales with the size of your environment. This page records what it costs, how it was measured, and what the numbers do not tell you.

Everything here is reproducible:

cargo bench -p rivetsnap-core

The benchmark source is crates/rivet-core/benches/collect.rs.

What is measured

collect/cold and collect/warm-unchanged both capture a synthetic tree of 2,000 files of about 1 KiB each, split across two directories — a wide, shallow shape, because that is what a real dotfile set looks like.

• cold — nothing is known about a previous capture. Every file is read in full, scanned for secrets, and written into the snapshot tree. This is what every snapshot cost before v0.8.
• warm-unchanged — the common case: a snapshot taken when nothing has changed since the last one. Each file is stream-hashed and nothing else.

Baseline (v0.8.0)

Recorded on an 8-core x86-64 Linux workstation, cargo bench, criterion’s median of 100 samples. RAYON_NUM_THREADS=1 forces the collector back to a single worker, which is how the parallel and serial columns below were taken from the same code.

Benchmark	Serial (1 thread)	Parallel (8 threads)	Speed-up
collect/cold	68.5 ms	24.9 ms	2.8×
collect/warm-unchanged	30.8 ms	11.7 ms	2.6×

End to end, a repeat snapshot of an unchanged tree went from 68.5 ms to 11.7 ms — 5.9× — from the two v0.8 changes together.

Two honesty notes about that figure:

• The serial column is not literally v0.7. v0.7 read each file once for the secret scan and then read it again through fs::copy; the current cold path writes from the buffer the scan already required. So the real v0.7 number was somewhat worse than 68.5 ms, and the 5.9× is an understatement of the release-over-release change rather than an inflation of it.
• These are absolute timings on one machine with a warm page cache. Compare them against each other, not against numbers from your own hardware.

Against the v0.8 target (“a large tracked set of ~10k files completes in seconds, not minutes”): extrapolating the cold figure gives roughly 125 ms for 10,000 files of this size. The target is met with a very large margin. A capture dominated by a handful of huge files is a different shape and is bounded by disk throughput, not by Rivet.

Where the time goes

Two changes account for the table above, both in crates/rivet-core/src/collect.rs:

1. Parallel per-file work. Walking the tree stays sequential — it is cheap — but the read, secret scan, and byte copy for each file run as a rayon parallel iterator. Results are collected as inert values and replayed in sorted order, so a parallel capture produces a byte-identical manifest to a serial one. See Determinism below.
2. Content-addressed skip. Each captured file records a BLAKE3 hash of its content in the manifest. The next snapshot stream-hashes the source file and, if the hash is unchanged and the blob is still in the tree, reuses it without reading the file into memory, scanning it, or copying its bytes.

Streaming the hash matters for the second one: change detection needs only a digest, so a multi-gigabyte tracked file must not become a multi-gigabyte allocation. Only a file that has actually changed is ever read in full.

Determinism

Performance work here is constrained by a rule that outranks it: the same tree must always produce the same snapshot. Concretely, and asserted by tests in collect.rs:

• Manifest entries, warnings, and audit lines are emitted in path order, not in completion order.
• A capture blocked by a detected secret names the lexically first offending file, so the same tree always fails with the same message rather than naming whichever file a worker happened to reach first.

Why skipping a file is not a safety hole

A skipped file is not re-scanned for secrets. That is only sound because identical bytes give an identical verdict from the same scanner — and a future release with stricter rules must not silently grandfather in a file the previous one let through.

The gate is policy::SECRET_SCAN_VERSION, recorded in each manifest as secret_scan_version. PriorCapture::from_manifest refuses to trust any hash recorded under a different generation, so tightening the scanner and bumping that constant makes the next snapshot re-read and re-scan everything. The invariant the snapshot format promises holds either way: every file in a snapshot has been cleared by the rules the running binary ships.

If you change what policy::secret_findings detects, or the SECRET_PATH_COMPONENTS list, bump SECRET_SCAN_VERSION in the same commit.

Known limitations

• The manifest hash is a capture-time optimisation, not an integrity check. Restore never consults it. It is not a guarantee that the blob in the snapshot still matches what was captured — if you edit the snapshot repository by hand, Rivet will not notice, and the next capture will happily reuse the blob you edited. Treat the repository as Rivet-owned.
• files/ is no longer wiped before each capture. Anything the capture does not re-affirm is pruned afterwards instead. That prune is what removes a file which stopped being tracked; it is a correctness requirement, not tidiness.
• Benchmarks are not a CI regression gate yet. They compile and run in CI, which catches “the benchmark no longer builds”, but nothing fails a build on a slow-down. Criterion’s saved baselines make that possible later; the noise floor on shared CI runners is the obstacle.
• Package discovery is not covered by these numbers. Snapshot wall-clock on a real machine is often dominated by shelling out to the native package manager, which Rivet does not control.