Skip to content

Fail open in consumer_has_unread when a consumer has no cursor rows - #429

Open
sergeivad wants to merge 1 commit into
ClickHouse:mainfrom
sergeivad:sergeivad/gate-fail-open-without-cursors
Open

Fail open in consumer_has_unread when a consumer has no cursor rows#429
sergeivad wants to merge 1 commit into
ClickHouse:mainfrom
sergeivad:sergeivad/gate-fail-open-without-cursors

Conversation

@sergeivad

Copy link
Copy Markdown

The bug

A MessagesGate configured without an explicit source list reads consumer_cursors only. The job it gates is the very job whose poll creates the first cursor row — so with zero rows the gate can never open, and the job never runs to create them.

A consumer reaches that state in two ordinary ways:

  • a fresh install, before the job has ever run;
  • any time cleanup_expired_consumer_cursors deletes every row after a quiet stretch longer than the cursor TTL.

The failure is silent: the job is simply skipped forever, and nothing in the logs says why.

The fix

With zero cursor rows, report unread whenever any messages exist at all. The job then runs once, its poll seeds cursors at MAX(rowid), and the check reverts to the normal cursor-based path.

Backlog is still skipped — "new consumers see only future messages" is delivery semantics and is unchanged. Consumers that already have at least one cursor row keep ignoring untracked sources exactly as before, so this only affects the state that was previously unrecoverable.

Tests

New coverage in tests/test_cron_gates.py and tests/test_db.py for both the zero-row and the established-consumer paths. Full suite green on top of current main.

A no-sources MessagesGate reads consumer_cursors only, and the job it
blocks is the very job whose poll creates the first cursor row — so with
zero rows the gate could never open. A consumer reaches that state on a
fresh install, and again whenever cleanup_expired_consumer_cursors
deletes every row after a quiet stretch longer than the cursor TTL.

With zero cursor rows, report unread whenever any messages exist at all:
the job runs once, its poll seeds cursors at MAX(rowid) (backlog still
skipped — "new consumers see only future messages" is delivery
semantics and unchanged), and the check reverts to cursor-based.
Established consumers (>=1 cursor row) still ignore untracked sources.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Sep 7, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants