Fix SQLite "database is locked" errors under concurrent queue workers#128
Open
simonhamp wants to merge 1 commit into
Open
Fix SQLite "database is locked" errors under concurrent queue workers#128simonhamp wants to merge 1 commit into
simonhamp wants to merge 1 commit into
Conversation
The runtime `nativephp` connection was built with no `transaction_mode`, `busy_timeout`, `journal_mode` or `synchronous`, relying instead on a one-shot `PRAGMA` after the config rewrite. Those PRAGMAs only affect the connection alive at boot; they are lost on reconnect and never reach the lazily-created connections in queue worker processes. More importantly, the connection fell back to Laravel's default DEFERRED transaction mode. The database queue pops jobs in a read-then-write transaction (lockForUpdate then mark reserved), and under DEFERRED the write-upgrade fails immediately with SQLITE_BUSY_SNAPSHOT when another worker commits in between — an error the busy timeout does not retry. Build the connection from the app's existing `sqlite` config and set the concurrency-critical keys so Laravel's SQLiteConnector applies them on every connect in every process. transaction_mode in particular has no PRAGMA equivalent and can only be set here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Apps that run background queue workers against the bundled SQLite database hit frequent
SQLite... database is lockedfailures, even with only one or two workers and even though WAL is enabled.Two issues in
NativeServiceProvider::rewriteDatabase():The
nativephpconnection relies on one-shot PRAGMAs.journal_modeandbusy_timeoutare set withDB::statement(...)right after the config rewrite, but they aren't part of the connection config array. Laravel'sSQLiteConnectoronly issues PRAGMAs it finds in the config, so on any reconnect — and for the lazily-created connections in each queue worker process — these settings are silently dropped.The connection falls back to
DEFERREDtransactions (the real culprit). The database queue pops a job in a read-then-write transaction (getNextAvailableJob()→lockForUpdate()→ mark reserved). UnderDEFERRED,BEGINstarts as a read and upgrades to a write on the first write statement; if another worker has committed in between, SQLite returnsSQLITE_BUSY_SNAPSHOTimmediately and the busy handler does not retry it (retrying could deadlock). WAL and a largebusy_timeoutcannot help here.transaction_modehas no PRAGMA equivalent — it controls how Laravel emitsBEGIN— so it can only be fixed in the connection config.Fix
Build the
nativephpconnection from the app's existingsqliteconnection config (so any developer tuning carries over) and set the concurrency-critical keys explicitly:transaction_mode→IMMEDIATE— takes the write lock atBEGIN, so pop transactions serialise and wait on the busy timeout instead of erroring withSQLITE_BUSY_SNAPSHOT.busy_timeout,journal_mode(WAL),synchronous(NORMAL) — now declared in config so the connector re-applies them on every connect in every process, not just the one alive at boot.All are env-overridable and default to the same effective behaviour as before (WAL, 5s busy timeout), plus the
IMMEDIATEandNORMALimprovements.The trailing
PRAGMA journal_mode=WAL/busy_timeoutstatements are now redundant but left in place to keep the diff minimal (WAL persists at the file level regardless).Notes
mainstill exhibits this — the connection array has notransaction_mode/busy_timeout/journal_mode/synchronous.🤖 Generated with Claude Code