Skip to content

docs: fix errors and improve consistency across the documentation#4132

Merged
d-v-b merged 31 commits into
zarr-developers:mainfrom
d-v-b:claude/zarr-python-docs-review-764989
Jul 9, 2026
Merged

docs: fix errors and improve consistency across the documentation#4132
d-v-b merged 31 commits into
zarr-developers:mainfrom
d-v-b:claude/zarr-python-docs-review-764989

Conversation

@d-v-b

@d-v-b d-v-b commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

A full critical review of the documentation with every finding verified by executing the runnable snippets against the current source, followed by a series of rendering and navigation improvements. All tests/test_docs.py pass, strict mkdocs build is clean, and all relative links and anchors were mechanically checked.

Correctness fixes

performance.md had three claims contradicted by the page's own live-rendered output:

  • The memory-order example claimed "Fortran order gives a better compression ratio" — in Zarr format 3, order affects in-memory layout only; stored bytes are identical (verified: both orders stored byte-identical data). The section is rewritten around what order actually does.
  • The write_empty_chunks conclusion was inverted: the emptiness check runs when the flag is False, so random-data writes are slower with False, not True.
  • "Zarr's default concurrency of 64" → 10 (and the dependent arithmetic), and threading.max_workers "defaults to CPU count" → None; both now consistent with the correct statements earlier on the same page.
  • The stale "MemoryStore cannot be pickled" claim is replaced with the real caveat (pickling copies the data).

Other errors:

  • storage.md: the explicit-S3-filesystem example passed the bucket's virtual-hosted URL as endpoint_url and pointed FsspecStore at the filesystem root; now a correct copyable pattern.
  • config.md: the "current default configuration" block showed order: 'F' because an earlier snippet in the same build session mutated the config; also codecs.bytes.namecodecs.bytes.
  • arrays.md: "three configuration options" → all five ArrayConfig fields documented; memory:// URLs were attributed to MemoryStore (they resolve to an fsspec-backed store when fsspec is installed); wrong column indices in two comments; stray doctest-output line removed from data_types.md.
  • v3_migration.md: Group.move and dimension_separator removed from the "not yet ported" lists (both are implemented); a truncated store-table row removed; the lapsed 2.x support window rewritten in past tense; the "🚧 Work in Progress 🚧" framing updated to 3.2.x reality.
  • cli.md: the zarr[cli] extra is now documented (a plain install crashes with ModuleNotFoundError); the dry-run example output corrected to the real file:// form.
  • installation.md: typing_extensions minimum corrected to 4.14; all five optional-dependency groups documented.
  • gpu.md: zarr.Config.enable_gpu (nonexistent) → zarr.config.enable_gpu(); installation guidance added.
  • contributing.md: docs build output is site/, not docs/_build/html; serve URL corrected; outdated doctest note removed.
  • extending.md: custom data types are supported today (links to the dtype guide and worked example), not "in the future".
  • quick-start.md: result="html" on a Python-repr block → result="code".
  • Standardized on American spelling in living prose (behaviourbehavior).

Rendering fixes

  • New MkDocs hook (mkdocs_hooks.py): fences marked test="true" or exec="false" (the repo's docs-validation convention) failed superfences parsing entirely and spilled their contents into the page as raw markdown — most visibly the custom dtype example, where the included script's PEP 723 header rendered as a cascade of headings. The GPU page, quick-start S3 example, and all exec="false" opt-outs were equally affected. The hook registers a second python fence, tried when Markdown Exec's declines, that renders these as ordinary highlighted code blocks.
  • Invisible code blocks: three exec="true" blocks without source="above" rendered as nothing (the rectilinear-chunks enable snippet in arrays.md, and two config snippets in performance.md). All are now visible.
  • data_types.md: two blocks lacked result formatting and injected their output as raw paragraphs.

Examples

  • Rectilinear chunks: converted from a mkdocs-jupyter notebook (which rendered its PEP 723 header as a code cell, leaked a ConfigSet repr, and didn't match the site style) to a markdown page in the same style as the custom dtype example, with captured outputs as static blocks and an admonition explaining the xarray-fork requirement. Page URL unchanged. With no notebooks left, the mkdocs-jupyter plugin and dependency are removed.
  • Custom dtype example: title shortened to "Custom Data Type"; README now leads with uv run (PEP 723) since plain python fails without deps installed.

API reference navigation

  • One nav entry per object, mirroring the flat zarr namespace: the create/open/save/load pages grouped functions by verb — an organization that doesn't exist in the codebase. Each of the 26 top-level functions now has its own page (api/zarr/functions/<name>.md) and nav entry, alphabetized case-insensitively alongside the classes and modules, so zarr.Array (class) sits next to zarr.array (function). Old URLs redirect.
  • Kind badges: every API nav entry shows the compact func/mod/class doc-symbol badge that mkdocstrings already uses in each page's table of contents, and identifiers render in monospace. Verified in the rendered site: no wrapped entries, badge CSS loads globally.
  • Previously undocumented public functions added to the reference: zarr.consolidate_metadata and zarr.print_debug_info. The unimplemented copy/copy_all/copy_store stubs and deprecated zarr.tree are deliberately excluded.
  • Fixed accidental nav labels: two literal "Index" entries, an "API" section nested under "API Reference", title-cased "Cpu"/"Gpu", and ambiguous twin "metadata" entries.

Usability improvements

  • User Guide nav reordered into a beginner→advanced progression.
  • quick-start.md: proper heading hierarchy, zarr[remote] note for the cloud example, and a "Next steps" section.
  • user-guide/index.md now lists the CLI, experimental-features, and examples pages.
  • groups.md: explains group/create_group/open_group; documents member enumeration and deletion.
  • attributes.md: documents update_attributes for bulk updates, deletion, and links consolidated metadata.
  • consolidated_metadata.md: documents the public use_consolidated=True/False/None knob.
  • glossary.md: adds Group, Attributes, and Consolidated Metadata entries; covers v2 metadata documents.
  • Cross-links added between related pages throughout; "Coming soon" stubs replaced with links to existing content.
  • experimental.md: CacheStore sections properly nested; links the experimental API policy.
  • release-notes.md: version headings and dates normalized.

Verification

  • tests/test_docs.py: 57 passed, 2 skipped (GPU/cupy and a known skip).
  • mkdocs build --strict: clean (validates every mkdocstrings directive and redirect target).
  • All relative links and anchors mechanically verified; codespell clean.
  • Nav and badge rendering verified by serving the built site and inspecting the DOM.

🤖 Generated with Claude Code

dependabot Bot and others added 14 commits May 31, 2026 19:28
…#176)

Bumps the actions group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi) | `0.9.5` | `0.9.6` |
| [codecov/codecov-action](https://github.com/codecov/codecov-action) | `6.0.0` | `6.0.1` |
| [github/issue-metrics](https://github.com/github/issue-metrics) | `4.2.2` | `4.2.7` |
| [j178/prek-action](https://github.com/j178/prek-action) | `2.0.3` | `2.0.4` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `7.0.0` | `7.0.1` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` |
| [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) | `1.13.0` | `1.14.0` |
| [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) | `0.5.3` | `0.5.6` |



Updates `prefix-dev/setup-pixi` from 0.9.5 to 0.9.6
- [Release notes](https://github.com/prefix-dev/setup-pixi/releases)
- [Commits](prefix-dev/setup-pixi@1b2de7f...5185adf)

Updates `codecov/codecov-action` from 6.0.0 to 6.0.1
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@57e3a13...e79a696)

Updates `github/issue-metrics` from 4.2.2 to 4.2.7
- [Release notes](https://github.com/github/issue-metrics/releases)
- [Commits](github-community-projects/issue-metrics@c9e9838...1e38d5e)

Updates `j178/prek-action` from 2.0.3 to 2.0.4
- [Release notes](https://github.com/j178/prek-action/releases)
- [Commits](j178/prek-action@6ad8027...bdca6f1)

Updates `actions/upload-artifact` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v7...043fb46)

Updates `actions/download-artifact` from 7.0.0 to 8.0.1
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...3e5f45b)

Updates `pypa/gh-action-pypi-publish` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](pypa/gh-action-pypi-publish@v1.13.0...cef2210)

Updates `zizmorcore/zizmor-action` from 0.5.3 to 0.5.6
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](zizmorcore/zizmor-action@b1d7e1f...5f14fd0)

---
updated-dependencies:
- dependency-name: prefix-dev/setup-pixi
  dependency-version: 0.9.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: codecov/codecov-action
  dependency-version: 6.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: github/issue-metrics
  dependency-version: 4.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: j178/prek-action
  dependency-version: 2.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/download-artifact
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: pypa/gh-action-pypi-publish
  dependency-version: 1.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.5.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
A full review of the documentation, verified by executing every runnable
snippet against the current source. Highlights:

Correctness:
- performance.md: fix three claims contradicted by the page's own rendered
  output (memory-order/compression, inverted write_empty_chunks conclusion,
  wrong async.concurrency and threading.max_workers defaults); update stale
  MemoryStore pickle claim
- Make three invisible exec blocks visible (rectilinear enable snippet in
  arrays.md, two config snippets in performance.md)
- storage.md: fix the explicit-S3-filesystem example (endpoint_url misuse,
  missing bucket path)
- config.md: show true defaults (earlier example no longer leaks array.order
  into the session); fix codecs.bytes key
- arrays.md: document all five ArrayConfig options; correct memory:// store
  description; fix wrong column-index comments; remove orphaned example
- v3_migration.md: remove Group.move and dimension_separator from
  not-yet-ported lists (both work); fix truncated store table row; update
  lapsed 2.x support-window and stale work-in-progress framing
- cli.md: document the required cli extra; fix dry-run output; name the
  equivalent functions in zarr.metadata.migrate_v3
- installation.md: correct typing_extensions minimum; document all optional
  dependency groups (remote, gpu, cli, optional, cast-value-rs)
- gpu.md: fix nonexistent zarr.Config.enable_gpu; add installation guidance
- contributing.md: fix docs build output dir and serve URL; drop outdated
  doctest note
- extending.md: custom data types are supported now, not future work
- data_types.md: remove stray doctest output from a source block; use public
  zarr.dtype import; drop dead warnings-suppression boilerplate
- quick-start.md: fix result="html" on a Python-repr output block

Usability:
- Reorder the user-guide nav into a beginner-to-advanced progression
- quick-start.md: add H1 and heading hierarchy, remote-storage install note,
  and a Next steps section
- user-guide/index.md: list the CLI, experimental, and examples pages
- groups.md: explain group/create_group/open_group; document member
  enumeration and deletion
- attributes.md: document bulk updates via update_attributes and deletion
- consolidated_metadata.md: document the public use_consolidated knob
- glossary.md: add Group, Attributes, and Consolidated Metadata entries;
  cover v2 metadata documents
- Add cross-links between related pages throughout; replace "Coming soon"
  stubs with links to existing content
- experimental.md: nest CacheStore sections properly; link the experimental
  API policy
- release-notes.md: normalize version headings and dates

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 8, 2026
@github-actions github-actions Bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 8, 2026
@d-v-b

d-v-b commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@zarr-developers/python-core-devs this is a big, flat, boring collection of AI-authored changes to our docs. If anyone wants to spot-check it, please have a look. Otherwise, I'm going to review everything by hand, and then merge shortly.

"id": "e8d42341-c242-44f5-ad6a-491370e3ffab",
"metadata": {},
"outputs": [],
"source": []

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed a dead cell

Comment thread docs/user-guide/arrays.md
)
print(f"Compressors: {z.compressors}")
```

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this example is redundant. we show None = no compression earlier.

Comment thread docs/user-guide/arrays.md
foo[:, :] = np.random.random((1000, 100))
bar[:] = np.arange(100)
print(root.tree())
```

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this example has nothing to do with block indexing

d-v-b and others added 6 commits July 8, 2026 22:15
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Markdown Exec's superfences fence only claims exec="true" blocks. Fences
carrying the repo's other validation markers (test="true", exec="false")
failed superfences validation entirely and their contents spilled into the
page as raw markdown -- most visibly the custom dtype example, where the
included script's PEP 723 header rendered as a series of headings. The GPU
page, the quick-start S3 example, and the exec="false" opt-out blocks were
equally affected.

Add an MkDocs hook that registers a second python fence, tried when
Markdown Exec's declines, which strips the validation attributes and
delegates to the standard superfences highlighter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The notebook rendered oddly via mkdocs-jupyter: the PEP 723 dependency
header appeared as the first code cell, a bare ConfigSet repr leaked into
the output, and the page styling did not match the rest of the site. Since
the example cannot execute at build time anyway (it requires an xarray
fork with rectilinear chunk grid support plus remote data), convert it to
a markdown page in the same style as the custom dtype example, with the
captured outputs shown as static blocks and an admonition explaining the
requirements.

The page URL is unchanged. With no notebooks left in the docs, also drop
the mkdocs-jupyter plugin and dependency.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
d-v-b and others added 7 commits July 9, 2026 10:22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The API nav was a flat list whose labels came from each page's first
mkdocstrings heading, mixing classes (Array), modules (codecs), and
function groups (create) with no visual distinction, plus accidental
labels like a title-cased Cpu, two literal Index entries, and an API
section nested under API Reference.

Group the nav into Classes / Functions / Modules with explicit titles:
classes as zarr.Array and zarr.Group, function pages as Creating/Opening/
Saving/Loading, and modules by their dotted import path (zarr.storage,
zarr.abc.store, ...), so the naming convention itself distinguishes the
kinds. Also give the four subpackage index pages consistent titles. Page
URLs are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Users browsing the API reference don't know in advance whether the thing
they want is a class, function, or module, so grouping the nav by kind
made them guess. Flatten the list back and annotate each entry with its
kind instead: zarr.Array (class), Creating (functions), zarr.storage
(module). Nesting is kept only where it means containment (the abc, api,
buffer, and testing subpackages); their children are unannotated since
context makes them modules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
create/open/save/load, as on main, rather than invented verb labels;
the (functions) annotation carries the kind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Material passes nav titles through as HTML, so wrap the identifier part
of each API reference nav label in <code>, keeping the (class)/(functions)/
(module) annotations in regular type. Verified the HTML is not escaped in
the sidebar, is stripped from the browser <title>, and behaves in search
like any backticked heading.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The create/open/save/load pages grouped functions by verb, an organization
that doesn't exist in the codebase -- all of these are top-level names in
the flat zarr namespace. Replace them with one page and one nav entry per
function (zarr.create_array, zarr.open_group, ...), alphabetized case-
insensitively alongside the classes and modules, so zarr.Array (class)
sits next to zarr.array (function).

This also adds previously undocumented public functions to the API
reference: zarr.consolidate_metadata and zarr.print_debug_info. The
unimplemented copy/copy_all/copy_store stubs and the deprecated zarr.tree
are deliberately excluded. Old create/open/save/load URLs redirect to
representative function pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spelled-out (function)/(module)/(class) annotations made 8 of 41
monospace nav entries wrap to two lines. Replace them with the compact
func/mod/class doc-symbol badges that mkdocstrings already renders in
every page's table of contents, joined to the name with a non-breaking
space so badge and name never separate. Verified in the rendered site:
no wrapped entries, badge CSS loads globally, and function-page titles
render as e.g. 'func zarr.open_group'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@d-v-b

d-v-b commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

updated the API navbar from this:

image

to this:

image

@d-v-b

d-v-b commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

I'm going to self-merge, these are all safe, minor improvements or corrections to the docs.

@d-v-b d-v-b merged commit 79486f7 into zarr-developers:main Jul 9, 2026
30 checks passed
@d-v-b d-v-b deleted the claude/zarr-python-docs-review-764989 branch July 9, 2026 09:49
@maxrjones

Copy link
Copy Markdown
Member

Hey, @d-v-b, I really appreciate your work improving our documentation. I'd like to suggest a couple process-based improvements. I'd be grateful if you take these requests in a way that doesn't devalue your overall work, which is keeping Zarr-Python moving forward and incredible valuable.

updated the API navbar from this:

image to this: image

I think this an acceptable change, but out of scope from "fix errors and improve consistency across the documentation". I would prefer that PRs remain neatly scoped. Taking this as an example, others could tune in to a PR with a description "docs: update API documentation TOC" if they care, whereas they may have unsubscribed already from this PR.

I think it's important for core devs to follow our contributing policies, or open a PR to change them. The relevant parts for this PR include:

  • PR descriptions being written by a human
  • The human having reviewed all changed before opening the PR

@d-v-b

d-v-b commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback max! My judgement was that changing the navbar to consistently class / function labels for all the contents was a safe fix, since it doesn't affect URLs and the old navbar was confusing in a way that seemed unintentional.

But I would be very surprised if anyone particularly interested in any specific part of the docs would tune in to this PR on the basis of its title -- this PR is tranche of unrelated fixes scattered across the docs, which is only possible because LLMs can translate prompts like "review our docs, find errors and opportunities for improvement" into concrete changes. Ideally these prompts result in the LLM finding no problems, but until then, I do think this is an efficient way to surface problems and fix them. This of course relies heavily on our judgment as maintainers -- I would only open a PR like this in a repo I am maintaining, and docs changes won't affect runtime behavior, and the docs are not turing complete, at least not how we use them, so changes don't interact, and a lot of changes are safe in one go.

Which brings me to the second point: if we accept that PRs like this have value (happy to discuss this but I think doing these changes in 1 PR is far more efficient than opening separate PRs for each change), then requiring a human-authored PR description feels like a waste of time. At this point, I view a PR like this (administered by someone very familiar with the codebase) as not so different from a dependabot PR. We don't require dependabot PR descriptions to be written by humans. Instead, we require the consequences of dependabot PRs to be owned by humans. So I wonder if we should either revise the "human-authored PR description" requirement, or introduce a mechanism for explicitly LLM-authored PRs, like dependabot PRs. I am not using LLM text here because I want to cheat the system and boost my credentials, i'm just busy and the LLM writes PR descriptions more consistently and exhaustively than I do.

@maxrjones

maxrjones commented Jul 9, 2026

Copy link
Copy Markdown
Member

Thanks for the feedback max! My judgement was that changing the navbar to consistently class / function labels for all the contents was a safe fix, since it doesn't affect URLs and the old navbar was confusing in a way that seemed unintentional.

But I would be very surprised if anyone particularly interested in any specific part of the docs would tune in to this PR on the basis of its title -- this PR is tranche of unrelated fixes scattered across the docs, which is only possible because LLMs can translate prompts like "review our docs, find errors and opportunities for improvement" into concrete changes. Ideally these prompts result in the LLM finding no problems, but until then, I do think this is an efficient way to surface problems and fix them. This of course relies heavily on our judgment as maintainers -- I would only open a PR like this in a repo I am maintaining, and docs changes won't affect runtime behavior, and the docs are not turing complete, at least not how we use them, so changes don't interact, and a lot of changes are safe in one go.

Which brings me to the second point: if we accept that PRs like this have value (happy to discuss this but I think doing these changes in 1 PR is far more efficient than opening separate PRs for each change), then requiring a human-authored PR description feels like a waste of time. At this point, I view a PR like this (administered by someone very familiar with the codebase) as not so different from a dependabot PR. We don't require dependabot PR descriptions to be written by humans. Instead, we require the consequences of dependabot PRs to be owned by humans. So I wonder if we should either revise the "human-authored PR description" requirement, or introduce a mechanism for explicitly LLM-authored PRs, like dependabot PRs. I am not using LLM text here because I want to cheat the system and boost my credentials, i'm just busy and the LLM writes PR descriptions more consistently and exhaustively than I do.

It sounds like you prefer the option from my earlier comment of changing the contributing guide (possibly through an exclusion for core developers). That's alright with me. There's a very minimal subset of our guide that I actually care strongly about for core devs, beyond the obvious understanding/agreeing with/owning the changes:

  • meaningful commit messages our git history, in particular the one-liners. I feel strongly that a view of https://github.com/zarr-developers/zarr-python/commits/main/ should signal high-quality, thoughtful development.
  • having at minimum the start of a PR description being written by a human, regardless of whether it's internally/externally authored. I think writing one sentence about why you think its worth prompting an agent, spending time reviewing/owning the PR, and adding it to other's feed (via pre-release reviews at this point). I think this is a super low bar of additional work over reviewing/owning a set of AI written changing.

@d-v-b

d-v-b commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

sounds good, I'll open a PR to update the developer guide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants