Eliminate S3 bucket access

[kltm]

Once the mechanisms have switched over and enough traffic is on https://semanticsql.berkeleybop.io/{db}.gz , we will turn off public access to the underlying S3 bucket.

As well, we have now setup a logger on the bbop-sqlite bucket so that we can see who the stragglers are in more detail.

See: #110

[kltm]

[bot] Comment posted by a Claude Code agent on behalf of @kltm.

Aggregate data from the bbop-sqlite access logger (~6 days, 2026-07-09 → 07-14), to help scope the straggler drain before removing public access:

• ~85% of egress bytes still hit the raw S3 URL; ~15% now goes through the CDN (semanticsql.berkeleybop.io) at 87–90% edge-cache-hit. The fronting works; consumers largely have not migrated yet.
• Consumer fingerprint: dominated by Python stdlib urllib (multiple 3.x versions) with 0% conditional GETs (every fetch is a full re-download), from hundreds of ephemeral IPs, ~half Microsoft/Azure — i.e. GitHub-Actions / cloud CI on OAK ≤0.7.1 (raw-S3 + urllib path). Matches the pre-0.7.2 code exactly.
• Biggest objects (raw URL): go.db.gz is the single largest, then chebi, mondo, rhea, pr, ncit, hp, ncbitaxon — each repeat-pulled by hundreds of distinct IPs (exactly what the CDN's shared cache collapses).
• Encouragingly, a large consumer is already on the CDN, so ≥0.7.2 / env-override adoption is starting.
• Separately, a Meta facebookexternalhit crawler is pulling multi-GB .db.gz files — not a real consumer; blockable independent of this effort.

Implication for lockdown: the raw URL still carries the majority with a live, identifiable OAK-ecosystem-CI consumer set, so removing AllUsers now would break them. The drain should accelerate as OAK ≥0.7.2 propagates — chiefly the ODK image bump (INCATools/ontology-development-kit#1354 / #1355) — and the direct hardcoders in #115 repoint. Suggest gating removal on this logger showing raw-URL object GETs fall off.

— Posted by Claude Code agent on behalf of @kltm.