compact: cache each archive's referenced objects to speed up analysis

[ThomasWaldmann]

analyze_archives() scans every archive's items on each run just to collect the object ids it references. Cache that per archive in the repo as cache/referenced-by-archive., so an unchanged archive is not scanned again next time; caches of archives that no longer exist are dropped.

The cache stores the referenced object ids (id -> plaintext size) plus the archive's source file count and content size, both counted per occurrence, so the reported statistics are identical to a full scan. Integrity is protected by a sha256 appended to the serialized blob.

[ThomasWaldmann]

Did a quick test:

• remote repository on google drive (SLOW)
• stats:

Overall statistics, considering all 5 archives in this repository:
Source data size was 213 GB in 2164621 files.
Deduplicated size is 47 GB.
Deduplication factor is 0.22.
Repository size is 42 GB in 281212 objects.
Compression factor is 0.88.

• compact runtime (no cache, before this change): 7:37
• compact runtime (2nd run, with cache): 2:04

[codecov]

Codecov Report

❌ Patch coverage is 85.71429% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.18%. Comparing base (bc3f14d) to head (398ff20).
⚠️ Report is 8 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/borg/archiver/compact_cmd.py	85.71%	9 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9896      +/-   ##
==========================================
- Coverage   85.32%   85.18%   -0.14%     
==========================================
  Files          93       93              
  Lines       15671    15793     +122     
  Branches     2387     2400      +13     
==========================================
+ Hits        13372    13454      +82     
- Misses       1598     1633      +35     
- Partials      701      706       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.