docs: consolidate pre-publish gate management via Inventor and MCP (replaces #750, #752)

[jordanc-relevanceai]

Summary

Documents managing pre-publish quality gates via Inventor or MCP, consolidated from two open PRs that both edited build/agents/build-your-agent/evals.mdx. Branched from main.

• Pre-publish gates via Inventor or MCP — how the gate check runs automatically when publishing programmatically, the four MCP tools for managing gate config, the skip_checks bypass, and the fail-closed behavior
• invent.mdx — capability-card update noting Inventor can manage gate configuration

Replaces these PRs (now closed)

PR	What it covered	Status here
#752	Pre-publish gates via Inventor/MCP (TSP-1372)	Folded in, with corrections below
#750	Custom Tool evaluator checks (TSP-1370)	Omitted entirely — see below

Omitted: everything from #750

evals-custom-tool-check (flag 785246) is at 0% rollout and has never been called. No customer can create a Custom Tool check, so the accordion, the FAQ entry, the billing note, the check-type list change, and the tools.mdx / create-a-tool.mdx cross-links are all left out rather than caveated. Same call #742 made on the eval cost-breakdown columns.

Worth keeping #750's text for when it ships — the contract it describes is accurate. Verified against CustomToolEvalRuleForm.vue:236-243: a conversation input, a boolean passed output, an optional reason string, and a Create evaluator tool scaffolding button. One correction to carry over: the code blocks saving the check when the tool drifts out of contract, it does not "fail until resolved" as #750 claimed.

Corrections to #752

All four MCP tool names were missing their prefix. #752 documented get_pre_publish_status, upsert_pre_publish_config, delete_pre_publish_config and poll_pre_publish_check. They are registered as relevance_get_pre_publish_status, relevance_upsert_pre_publish_config, relevance_delete_pre_publish_config and relevance_poll_pre_publish_check (apps/nodeapi/src/mcp/tools/tool-names.ts). Fixed — as written, none of them would have worked.

Dropped the flag key, and the rollout caveat with it. #752 named eval-pre-publish-per-publish-per-test-set-be. No such flag exists — the real key is eval-pre-publish-per-test-set-be (flag 748191), and it is at 100% rollout. I initially replaced the key with a "rolling out progressively, contact your account manager" note; that was also wrong, so it's now removed entirely. The feature is fully available.

Verified as accurate and kept

• The 1–10 test set range on upsert_pre_publish_config — MAX_CONFIG_TEST_SETS = 10 with .min(1) (apps/nodeapi/src/mcp/tools/eval-pre-publish.ts:26,48-50).
• skip_checks exists as a bypass parameter (apps/nodeapi/src/mcp/tools/agent.ts, workforce.ts).
• Per-test-set thresholds — each entry carries threshold_score (0–100) and block_on_failure.

Test plan

• Mintlify preview renders the pre-publish gates section under Publish, above Cost and billing
• Check types accordion group and check-creation step 3 are unchanged from main
• invent.mdx capability card renders

Note on evals.mdx contention

This page also has #742 and #747 open against it. This PR only touches the Publish section, so it should not conflict with either. #747 is separately blocked on a 0% flag.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
relevanceai	🟢 Ready	View Preview	Aug 5, 2026, 4:25 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[github-actions]

🎯 Vibe check

Reviewed: 4 files (2 with issues, 2 clean)

Scores

	Dimension	Score	What's holding it back
🔴	Consistency	4/10	tools.mdx has 9 heading-capitalization violations and 3 uses of the banned word "powerful". evals.mdx has one bold-text-as-section-header.
🟡	Technical clarity	7/10	tools.mdx images have no alt text; the "Let agent decide" input-mode description is vague; "Common Tool Categories" is too generic to be actionable.
🟡	Non-technical clarity	7/10	tools.mdx opens with marketing language ("powerful digital assistant") and closes the same way. evals.mdx and invent.mdx are well-grounded and reader-friendly.
🟡	Structure	6/10	tools.mdx mixes bold-as-heading with real headings, uses a plain numbered list where <Steps> would help, and has a "Common Tool Categories" section that reads as filler. Everything else is structured well.

Score key: 🟢 9–10, 🟡 6–8, 🔴 1–5. Scores are a single overall judgment about the whole PR — not per file.

✨ Overall vibe: evals.mdx, invent.mdx, and create-a-tool.mdx are solid — clear structure, correct terminology, good component usage, and useful cross-references between the evals and tools systems. tools.mdx is the weak link: it carries a full deck of heading-capitalization violations and three banned-word hits that need fixing before merge.

🔧 Issues (16)

• build/agents/build-your-agent/tools.mdx:8 — powerful is a banned word: "tools transform your agent from a conversational interface into a powerful digital assistant"
• build/agents/build-your-agent/tools.mdx:57 — powerful is a banned word: "Create powerful workflows by combining multiple tools in sequence"
• build/agents/build-your-agent/tools.mdx:71 — powerful is a banned word: "you can create a powerful assistant that delivers exceptional value"
• build/agents/build-your-agent/tools.mdx:12 — heading ## Adding Tools to Your Agent → sentence case: ## Adding tools to your Agent
• build/agents/build-your-agent/tools.mdx:20 — heading ## Tool Input Configuration Options → ## Tool input configuration options
• build/agents/build-your-agent/tools.mdx:41 — heading ## Guiding Your Agent's Tool Usage → ## Guiding your Agent's tool usage
• build/agents/build-your-agent/tools.mdx:51 — heading ## Best Practices for Tool Implementation → ## Best practices for tool implementation
• build/agents/build-your-agent/tools.mdx:62 — heading ## Common Tool Categories → ## Common tool categories
• build/agents/build-your-agent/tools.mdx:73 — heading ## Tool Approvals → ## Tool approvals
• build/agents/build-your-agent/tools.mdx:77 — heading ### Auto Run → ### Auto run
• build/agents/build-your-agent/tools.mdx:83 — heading ### Approval Required → ### Approval required
• build/agents/build-your-agent/tools.mdx:89 — heading ### Let Agent Decide → ### Let Agent decide
• build/agents/build-your-agent/tools.mdx:95 — heading ## Tool Settings → ## Tool settings
• build/agents/build-your-agent/tools.mdx:10 — image has no alt text: ![]() → add a description of what the screenshot shows
• build/agents/build-your-agent/tools.mdx:22 — image has no alt text: ![]() → add a description of what the screenshot shows
• build/agents/build-your-agent/evals.mdx:113 — **Tool contract requirements** is bold text acting as a section header; CLAUDE.md says to use a proper heading instead. Change to ##### Tool contract requirements (or #### depending on desired hierarchy under the Custom Tool accordion).

🧩 Component suggestions (2)

• build/agents/build-your-agent/tools.mdx:14–18 — The three steps for adding a tool to an agent are a plain numbered list. Since they represent a sequential procedure with a clear start and end, <Steps> would add the visual progress indicator readers expect for procedural content. Replace with a <Steps> block: Step 1 "Select a tool", Step 2 "Add a custom tool", Step 3 "Configure usage settings".
• build/agents/build-your-agent/tools.mdx:43–47 — **Core Instructions**: followed by a nested bullet list uses bold text as a section label, which CLAUDE.md flags as a pattern to avoid. Either promote it to a ### heading (### Core instructions) or convert to a <Tip> callout with the three bullets written as a short paragraph — the content is brief enough that a callout fits.

🏗️ Page structure (1)

• build/agents/build-your-agent/tools.mdx:62–70 — "Common Tool Categories" lists five generic categories (Information Retrieval, Communication, Data Processing, Integration, Content Creation) prefaced with "while the specific tools available depend on your implementation." That caveat undercuts the section's usefulness, and the categories don't link anywhere or help the reader take an action. Consider removing this section and linking to the Integrations or Marketplace page instead, or replacing it with a list of actual available tools.

✅ Clean files (2)

build/invent/invent.mdx, build/tools/create-a-tool.mdx

🔋 Credit usage

Item	Count
Files reviewed	4
Context pages read	2
Total lines processed	~1,172

Files read: build/agents/build-your-agent/evals.mdx (487 lines), build/agents/build-your-agent/tools.mdx (118 lines), build/invent/invent.mdx (241 lines), build/tools/create-a-tool.mdx (103 lines), get-started/core-concepts/tools.mdx (101 lines), build/agents/build-your-agent/triggers.mdx (122 lines)

[jordanc-relevanceai]

Going to be included in and superseded by a Evals documentation update shortly