feat: add CitationConsistencyChecker — deterministic, zero-token RAG citation validator

[Palo-Alto-AI-Research-Lab]

Addresses #10973, proposed by @DYNOSuprovo, who asked for a runtime groundedness/verification guardrail that runs locally without external API calls.

What this adds

A new CitationConsistencyChecker validator (haystack.components.validators) plus a small Citation dataclass. It sits after a Generator that produces answers with explicit quote-style citations and, using no model and no tokens, checks that each quoted passage appears verbatim in the retrieved Document it is attributed to. Each citation is routed to a consistent or inconsistent output with its status filled in.

It catches the three failure modes an LLM judge is worst at, precisely because they read as fluent and supportive:

• fabricated — a plausible quote that appears in no retrieved document;
• frankenquote — every word is real, but the sentence was stitched together and never written (the match is contiguous, not bag-of-words, so this fails);
• misattributed — a real quote lifted from a different retrieved document.

Why deterministic, and why complementary rather than competing

@DYNOSuprovo raised the "lost-in-the-middle" degradation on the LLM-based approach in #11031 (checking claims against entire document payloads). Because this component is pure normalized-substring matching, it does not read a long context and cannot be sweet-talked by a persuasive citation — it does not degrade with context length and needs no external API, which is exactly the local-verification property that was requested. It is intended as a cheap first stage in front of an LLM groundedness checker (such as #11031), not a replacement: the deterministic gate rejects the impossible-to-support quotes for free, so only genuinely ambiguous "does this real quote support the claim?" cases need a model.

API

from haystack.components.validators import CitationConsistencyChecker, Citation
from haystack.dataclasses import Document

docs = [Document(id="d1", content="Body weight was unchanged in both arms of the trial.")]
checker = CitationConsistencyChecker()
result = checker.run(
    citations=[
        Citation(claim="Weight did not change.", document_id="d1", quote="Body weight was unchanged in both arms"),
        Citation(claim="Weight dropped sharply.", document_id="d1", quote="Body weight fell dramatically"),
    ],
    documents=docs,
)
# result["consistent"]   -> first citation  (status="found")
# result["inconsistent"] -> second citation (status="not_found")

An optional treat_misattributed_as_consistent flag controls whether a quote found in the wrong retrieved document is tolerated (default: no).

Tests & tooling

• 13 unit tests covering found / frankenquote / fabricated / misattributed / empty-quote fail-closed / unknown-doc-id / no-content docs / mixed-batch routing / to_dict / from_dict / pipeline serialization roundtrip. All green.
• ruff check and ruff format clean; lazy import wired in validators/__init__.py; release note added.

Note on provenance

The standalone, framework-agnostic version of this gate — plus an optional burden-of-proof LLM judge for the ambiguous sufficiency case — is maintained here: https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate . This PR inlines only the deterministic gate so the component keeps Haystack’s zero-extra-dependency contract.

[vercel]

@antondziatkovskii is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[HaystackBot]

Hi @Palo-Alto-AI-Research-Lab, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

[HaystackBot]

Thanks for signing the CLA, @Palo-Alto-AI-Research-Lab! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

[anakin87]

Thank you for taking the time to contribute to Haystack.

After reviewing the PR, my impression is that the new dataclass + component do not provide enough additional value to justify the future maintenance effort, so I am closing this PR.

If you'd like to contribute to Haystack, Contributions wanted issues are a good place to find ideas that are currently a priority for the project.