docs: dev note for workflow chaining

[andreatnvidia]

📋 Summary

Adds a dev note and runnable recipe for workflow chaining as a pause/inspect/resume control surface. The post focuses on what named stage boundaries enable, including human review, reusable intermediate artifacts, downstream resume, and future workflow shapes.

🔗 Related Issue

N/A

🔄 Changes

• Adds the "Pause, Inspect, Resume" dev note with hero/diagram/result assets for the workflow chaining story.
• Adds a headless document review gate recipe that runs to review_candidates, writes a reviewed artifact, and resumes downstream from that artifact.
• Adds the recipe page, recipe card, dev note card, and Fern navigation entry.
• Adds tests covering page generation, review artifact creation, and the end-to-end recipe output.
• Compresses large PNG assets below the pre-commit 600 KB limit; the workflow chaining diagram now uses a clean no-grid background.

🧪 Testing

• .venv/bin/ruff check .
• .venv/bin/ruff format --check .
• .venv/bin/pytest packages/data-designer/tests/docs/test_document_review_gate_recipe.py
• env npm_config_cache=/private/tmp/npm-cache-datadesigner make check-fern-docs (0 errors, 1 existing Fern warning)

✅ Checklist

• Follows commit message conventions
• Commits are signed off (DCO)
• Architecture docs updated (N/A - docs/recipe only)

[github-actions]

Fern preview: https://nvidia-preview-pr-775.docs.buildwithfern.com/nemo/datadesigner

Fern previews include the docs-website version archive with PR changes synced into latest. Notebook tutorials are rendered without execution outputs in previews.

[nabinchha]

Something seems off here....

[andreatnvidia]

This is GitHub's rendered diff resolving Fern's root-relative /assets/... path against github.com. The tracked image is valid and renders in the Fern preview, so I kept the established Fern asset path.

[greptile-apps]

Greptile Summary

This PR adds a "Pause, Inspect, Resume" developer note and a headless document review gate recipe that demonstrates workflow chaining as a control surface. It also moves ResumeMode from artifact_storage.py to the public data-designer-config package, wiring it into the lazy-import registry and updating all import sites.

• Recipe (document_review_gate.py): Generates synthetic invoice/form pages, runs the workflow to a named review_candidates stage, writes a simulated reviewed artifact, and resumes downstream via stage_output_overrides — covering the full pause/inspect/resume lifecycle with accompanying tests.
• ResumeMode refactor: Moves the enum to run_config.py and re-exports it via dd.ResumeMode, making it part of the stable public API with a matching export test.
• Docs: Adds the devnote MDX, recipe page, navigation entries in fern/versions/latest.yml, and compressed PNG assets.

Confidence Score: 4/5

Safe to merge after fixing the --review-pages CLI validation gap.

The recipe explicitly fixed --num-records validation at the CLI boundary, but --review-pages was left with plain type=int. Passing --review-pages 0 or -1 is accepted by argparse and propagates to ReviewSelectionParams(max_review_pages=0), which Pydantic immediately rejects with a ValidationError — an inconsistent and confusing failure for a documented runnable example. Everything else — the ResumeMode refactor, import wiring, documentation, and tests — is correct and complete.

docs/assets/recipes/workflow_chaining/document_review_gate.py — the build_arg_parser function at the --review-pages argument definition.

Important Files Changed

Filename	Overview
docs/assets/recipes/workflow_chaining/document_review_gate.py	825-line recipe implementing the workflow-chaining review gate pattern; one concrete validation gap: --review-pages accepts non-positive integers via plain type=int while the downstream Pydantic model enforces ge=1, causing a confusing crash instead of a clean CLI error.
packages/data-designer-config/src/data_designer/config/run_config.py	Adds ResumeMode StrEnum (NEVER, ALWAYS, IF_POSSIBLE) to the config package; clean move from artifact_storage.py.
packages/data-designer-config/src/data_designer/config/init.py	Exports ResumeMode via the lazy-import registry and TYPE_CHECKING block; consistent with the existing pattern for other config types.
packages/data-designer-engine/src/data_designer/engine/storage/artifact_storage.py	Removes the local ResumeMode definition and replaces it with an import from data_designer.config.run_config; no logic changes.
packages/data-designer/src/data_designer/interface/init.py	Updates TYPE_CHECKING import and lazy-import entry for ResumeMode to point to the new canonical location in run_config.
packages/data-designer/tests/docs/test_document_review_gate_recipe.py	Comprehensive integration tests covering page generation, metadata schema, box bounds, review artifact writing, end-to-end recipe output, overwrite guard, and CLI rejection of non-positive --num-records; --review-pages validation gap is not yet covered.
packages/data-designer-config/tests/config/test_run_config.py	Adds one test asserting dd.ResumeMode is ResumeMode, verifying the public re-export is wired correctly.
fern/versions/latest.yml	Adds Workflow Chaining recipe section and Pause, Inspect, Resume devnote to the Fern navigation; no structural issues.
fern/versions/latest/pages/devnotes/posts/annotate-hard-pages-review-gates.mdx	241-line devnote explaining workflow chaining as a control surface with diagrams, comparison tables, and inline code snippets; technically accurate relative to the recipe.
fern/versions/latest/pages/recipes/workflow_chaining/document_review_gate.mdx	Recipe page with run instructions and a minimal code excerpt illustrating the pause/resume boundary; content matches the actual recipe script.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CLI as CLI / main()
    participant RR as run_recipe()
    participant RS as run_to_review_stage()
    participant BW as build_workflow()
    participant WF as workflow.run()
    participant WS as write_simulated_review_artifact()
    participant RF as resume_from_reviewed()

    CLI->>RR: count, seed, review_pages, overwrite
    RR->>RS: count, seed, review_pages
    RS->>BW: build stages (document_pages → review_candidates)
    BW->>WF: "targets=review_candidates"
    WF-->>RS: WorkflowResults
    RS->>RS: results.export_stage(review_candidates, path)
    RS-->>RR: review_path

    RR->>WS: base_dir
    WS->>WS: read review_candidates parquet
    WS->>WS: fill human_boxes for selected rows
    WS-->>RR: reviewed_path

    RR->>RF: count, seed, review_pages
    RF->>BW: rebuild stages (same params)
    BW->>WF: "resume=ALWAYS, stage_output_overrides"
    WF-->>RF: WorkflowResults
    RF->>RF: results.export_stage(final_dataset, path)
    RF-->>RR: final_path
    RR-->>CLI: final_path

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CLI as CLI / main()
    participant RR as run_recipe()
    participant RS as run_to_review_stage()
    participant BW as build_workflow()
    participant WF as workflow.run()
    participant WS as write_simulated_review_artifact()
    participant RF as resume_from_reviewed()

    CLI->>RR: count, seed, review_pages, overwrite
    RR->>RS: count, seed, review_pages
    RS->>BW: build stages (document_pages → review_candidates)
    BW->>WF: "targets=review_candidates"
    WF-->>RS: WorkflowResults
    RS->>RS: results.export_stage(review_candidates, path)
    RS-->>RR: review_path

    RR->>WS: base_dir
    WS->>WS: read review_candidates parquet
    WS->>WS: fill human_boxes for selected rows
    WS-->>RR: reviewed_path

    RR->>RF: count, seed, review_pages
    RF->>BW: rebuild stages (same params)
    BW->>WF: "resume=ALWAYS, stage_output_overrides"
    WF-->>RF: WorkflowResults
    RF->>RF: results.export_stage(final_dataset, path)
    RF-->>RR: final_path
    RR-->>CLI: final_path

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
docs/assets/recipes/workflow_chaining/document_review_gate.py:807
`--review-pages` uses plain `type=int`, so `--review-pages 0` or `--review-pages -1` passes argparse validation and reaches `_review_candidates_builder(review_pages)`, which immediately instantiates `ReviewSelectionParams(max_review_pages=0)`. Pydantic rejects that value because of `ge=1`, producing a cryptic `ValidationError` instead of the clean exit-code-2 CLI error the codebase already enforces for `--num-records`.

```suggestion
    parser.add_argument("--review-pages", type=_positive_int, default=3)
```

_{Reviews (5): Last reviewed commit: "Merge branch 'main' into andreatgretel/d..." | Re-trigger Greptile}

[github-actions]

Thanks for putting this together, @andreatgretel — this is a genuinely nice dev note, and the runnable recipe makes the workflow-chaining story concrete.

Summary

This is a docs + recipe PR: it adds the "Pause, Inspect, Resume" dev note, a headless document_review_gate recipe that runs a workflow to a named review_candidates stage, exports the intermediate artifact, simulates an external review, and resumes downstream via stage_output_overrides. It also wires up the Fern navigation, recipe card, dev-note card, a redirect, compressed PNG assets, and behavioral tests. The implementation matches the stated intent, and I verified the workflow-chaining APIs it leans on (compose_workflow, add_stage, run(targets=…, resume=…, stage_output_overrides=…), export_stage, ResumeMode, custom_column_generator, CustomColumnConfig) all exist and are used with correct signatures.

Findings

Suggestions — Take it or leave it

docs/assets/recipes/workflow_chaining/document_review_gate.py:497-499 — Fallback indexes into ground_truth_boxes[0] without a guard

• What: When every proposed box is dropped (the signature-skip branch removes the only box), the fallback does parse_boxes(row["ground_truth_boxes"])[0]. If a row ever had zero ground-truth boxes, this would raise IndexError.
• Why: In this recipe every generated page always has several labeled boxes, so it can't trigger today — but as a reusable, copy-pasteable recipe it's the kind of edge a reader might hit after swapping in their own seed data.
• Suggestion: A one-line guard (e.g. skip the row or fall back to [] when ground_truth_boxes is empty) would make the pattern safer to lift. Optional given the demo scope.

docs/assets/recipes/workflow_chaining/document_review_gate.py:119 — _font return annotation is slightly imprecise

• What: The annotation is ImageFont.ImageFont, but ImageFont.truetype(...) returns a FreeTypeFont and only the load_default() fallback returns ImageFont.
• Why: Purely cosmetic — callers only use draw.text/textbbox, which accept both. Flagging only for accuracy.
• Suggestion: Leave as-is, or widen to the shared base if you want it exact. Not worth churn.

fern/docs.yml:37 — Redirect from a slug that was never published

• What: The new redirect maps /dev-notes/annotate-the-hard-pages → /dev-notes/pause-inspect-resume. Since this dev note is brand new (its slug resolves to pause-inspect-resume), there doesn't appear to be a prior public URL at annotate-the-hard-pages.
• Why: A redirect for a URL that was never live is harmless but adds a little config noise and can be confusing later ("what used to be here?").
• Suggestion: If the post was previously shared under that slug (draft/preview link), keep it; otherwise it can probably be dropped. Your call.

What Looks Good

• The dev note is well-structured and honest — the "Why not just run two workflows?" table draws the operational distinction clearly, and the recipe stays "deliberately ordinary Data Designer," which reinforces the declarative contract nicely.
• Tests verify behavior, not plumbing — asserting image dimensions, metadata validity, box-in-bounds geometry, selected-row counts, and source == "human_review" counts through the full run is exactly the right altitude for a recipe test. The spec_from_file_location loader for a non-importable script is a clean touch.
• Slug/nav/card consistency checks out — the BlogCard href (/dev-notes/pause-inspect-resume), the recipe card, the nav entry, and the recipe-page cross-link all line up with Fern's title-derived slugs.

Notes

• I couldn't re-run ruff check / ruff format --check in this CI environment (ruff isn't on PATH here). The PR checklist indicates they passed locally, and CI will enforce them regardless.
• Two existing inline comments already raise (a) whether ResumeMode should live in data_designer.config so recipes can use dd.ResumeMode, and (b) whether conditional generation could achieve the same result within one stage. I'm not duplicating those — they're worth a reply from you, but neither blocks this docs/recipe PR.

Structural Impact (graphify, 2.4s)

Risk: LOW (localized change)

• 2 Python files, 0 AST entities, 0/82 clusters

This confirms a localized, additive change with no god-node or cross-package blast radius — consistent with a docs + standalone-recipe PR that imports the public API surface without modifying it.

Verdict

Ship it (with nits) — Only optional suggestions above; nothing blocking. Worth a quick reply to the two existing inline comments before merge, but the change itself is in good shape.

---

This review was generated by an AI assistant.

[andreatnvidia]

Addressed the remaining actionable bot feedback:

• Metadata validation now rejects empty ground-truth box sets with a page-specific error.
• Removed the redirect that was introduced with the unpublished article.
• Kept the _font annotation unchanged because it is cosmetic and widening only the return annotation would remain incomplete.

Validation: 294 tests passed, Ruff passed, and Fern reported 0 errors with 1 existing warning.

[nabinchha]

Following up on the workflow-chaining snippets — one small readability suggestion on the opening example. Not blocking, take it or leave it.

Workflow Chaining In One Minute — anchor the two "magic" names

fern/versions/latest/pages/devnotes/posts/annotate-hard-pages-review-gates.mdx (the first code block)

The snippet reads really well, but two names appear without introduction, and one is genuinely ambiguous:

import data_designer.config as dd

workflow = data_designer.compose_workflow(name="quality-gated-candidates")
workflow.add_stage("draft_rows", draft_rows_builder, num_records=1_000)
workflow.add_stage("quality_gate", quality_gate_builder)
workflow.add_stage("final_dataset", final_dataset_builder)

• data_designer looks like the module, but it's a DataDesigner instance. The only import shown is import data_designer.config as dd, and import a.b as c binds only c — it doesn't bind data_designer. So a reader who copies this hits a NameError, and the name visually collides with the data_designer package. Worth showing (or noting) where the instance comes from.
• *_builder are opaque. This is the post's central point — a stage is just an ordinary Data Designer config (the prose says exactly that right above the snippet) — but the snippet hides it behind bare names, so the reader never sees that a stage is a normal DataDesignerConfigBuilder they already know how to write.

Option A — lightest touch (a comment):

import data_designer.config as dd
from data_designer.interface import DataDesigner

# data_designer: a configured DataDesigner instance
# each *_builder: an ordinary DataDesignerConfigBuilder — a normal DD config wired in as a named stage
data_designer = DataDesigner(model_providers=[...])

workflow = data_designer.compose_workflow(name="quality-gated-candidates")
workflow.add_stage("draft_rows", draft_rows_builder, num_records=1_000)
...

Option B — make one builder concrete so the "it's just a normal DD config" idea lands. If you go this route, CustomColumnConfig mirrors what this recipe's stages actually do and avoids needing a model_alias + model config (an LLMTextColumnConfig would require model_alias=...). Something like:

draft_rows_builder = dd.DataDesignerConfigBuilder(model_configs=[])
draft_rows_builder.add_column(
    dd.CustomColumnConfig(name="draft", generator_function=draft_rows, ...)
)
# quality_gate_builder / final_dataset_builder are the same idea: ordinary builders.

Either way, the two takeaways for the reader are: where data_designer comes from, and that the stages are regular Data Designer configs wired into named stages.

Same pattern in the recipe page

fern/versions/latest/pages/recipes/workflow_chaining/document_review_gate.mdx has the same shape — imports dd, then uses an undefined workflow / write_simulated_review_artifact / base_dir. That one reads more clearly as an excerpt, so it bothers me less, but whatever convention you pick (self-contained vs. explicitly-an-excerpt) is worth applying to both so they feel consistent.

---

This review was generated by an AI assistant.

[nabinchha]

Approving — nice work on this one. The dev note is well-written and the runnable recipe backs the story up. I verified the workflow-chaining API in the snippets against the source (compose_workflow, add_stage, run(targets=/resume=/stage_output_overrides=), export_stage, dd.ResumeMode, dd.SkipConfig(when=...)) and ran the recipe tests — all 10 pass against the branch. The ResumeMode move into data_designer.config is a clean, correctly-layered refactor (single definition, engine imports from config, interface re-exports).

My comments above (snippet readability, and the earlier notes on unused image assets + the now-stale "docs only" line in the PR description) are all non-blocking — feel free to take or leave them.

[andreatnvidia]

@nabinchha thanks! Addressed your comments above.

[greptile-apps]

--review-pages uses plain type=int, so --review-pages 0 or --review-pages -1 passes argparse validation and reaches _review_candidates_builder(review_pages), which immediately instantiates ReviewSelectionParams(max_review_pages=0). Pydantic rejects that value because of ge=1, producing a cryptic ValidationError instead of the clean exit-code-2 CLI error the codebase already enforces for --num-records.

Suggested change

	parser.add_argument("--review-pages", type=int, default=3)
	parser.add_argument("--review-pages", type=_positive_int, default=3)

Prompt To Fix With AI

This is a comment left during a code review.
Path: docs/assets/recipes/workflow_chaining/document_review_gate.py
Line: 807

Comment:
`--review-pages` uses plain `type=int`, so `--review-pages 0` or `--review-pages -1` passes argparse validation and reaches `_review_candidates_builder(review_pages)`, which immediately instantiates `ReviewSelectionParams(max_review_pages=0)`. Pydantic rejects that value because of `ge=1`, producing a cryptic `ValidationError` instead of the clean exit-code-2 CLI error the codebase already enforces for `--num-records`.

```suggestion
    parser.add_argument("--review-pages", type=_positive_int, default=3)
```

How can I resolve this? If you propose a fix, please make it concise.