Skip to content

fix(models): apply defaults after concurrent inserts - #2276

Open
fzlzjerry wants to merge 5 commits into
tortoise:developfrom
fzlzjerry:fix/2060-update-or-create-race
Open

fix(models): apply defaults after concurrent inserts#2276
fzlzjerry wants to merge 5 commits into
tortoise:developfrom
fzlzjerry:fix/2060-update-or-create-race

Conversation

@fzlzjerry

Copy link
Copy Markdown

Description

Retry the existing transactional update path when another caller wins the initial insert. The losing update_or_create() call now applies its own defaults to a freshly read row rather than returning the other caller's values unchanged.

The creation transaction boundary and get_or_create() behavior stay unchanged. An explicit None check also keeps falsy model instances on the update path.

Motivation and Context

Closes #2060.

Re-reading the row matters: the unlocked conflict lookup may already be stale or its row may have been deleted. The retry uses the normal update path and its existing backend locking behavior, without adding backend-specific upsert SQL or changing the public API.

How Has This Been Tested?

  • The unchanged upstream CI workflow on the fork passed all 23 jobs on 45676b9: Python 3.10–3.14 across SQLite, PostgreSQL (asyncpg/psycopg), MySQL (InnoDB/MyISAM and asyncmy), and SQL Server, plus checks and coverage.
  • Nine regression/control cases cover concurrent defaults, disjoint fields, a stale conflict lookup, a deleted winner, caller transaction rollback, falsy instances, unrelated integrity errors, and unchanged get_or_create() semantics. Final fixtures yield six failures and three passing controls against the original implementation.
  • The race tests call the real creation helper and synchronize before its INSERT; no success/failure result is fabricated. Their unique name is non-nullable so SQL Server enforces the constraint as well.
  • Independent runs without method instrumentation passed 78 concurrent API calls on each of SQLite, PostgreSQL, and MySQL.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation. N/A — behavior correction, no API change.
  • I have updated the documentation accordingly. N/A.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Retry the existing transactional update path when a concurrent caller
wins creation. Re-read the row instead of saving the unlocked, possibly
stale conflict lookup result. Use an explicit None check so falsy model
instances still take the update path.

Keep get_or_create and the existing creation transaction boundary
unchanged. Cover concurrent defaults, stale/deleted rows, transaction
rollback, falsy models, and unrelated integrity errors.

Fixes tortoise#2060
SQL Server omits inline uniqueness for nullable fields, so the existing
UniqueName fixture never entered conflict recovery there. Use a dedicated
model with a non-null unique name instead of skipping the concurrency
regressions on SQL Server.
Copilot AI lite review requested due to automatic review settings September 9, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codspeed-hq

codspeed-hq Bot commented Sep 11, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 24 untouched benchmarks


Comparing fzlzjerry:fix/2060-update-or-create-race (d87b10b) with develop (e5bddf2)

Open in CodSpeed

@waketzheng

Copy link
Copy Markdown
Contributor

Thanks for the update — the fix logic is sound and the test coverage remains excellent. I have one design concern about the retry loop that I'd like to discuss before merging.

Concern: unbounded while True retry loop

The new implementation wraps the entire "locked read → update / create" flow in while True (line 210). The logic is correct: if _create_or_get() returns created=False (meaning a concurrent caller won the insert race), the loop restarts and retries the update path with a fresh read.

However, while True has no retry limit and no backoff. In theory, if _create_or_get() keeps returning created=False — e.g., because concurrent callers keep winning the insert race before this caller can enter the update path — the loop could spin indefinitely. In practice this is extremely unlikely, but an unbounded loop in a core ORM method is still a hazard worth addressing.

Suggested alternatives:

  1. Bounded retry with backoff: Replace while True with for attempt in range(max_retries) and add a small backoff (e.g., asyncio.sleep(backoff * (2 ** attempt))). Raise a clear exception after exhausting retries.
  2. Single retry is probably enough: The conflict-recovery path itself is already a "retry." In most cases, one retry should be sufficient to enter the update path. Consider capping at 1–2 attempts instead of an unbounded loop.
  3. If you believe infinite looping is mathematically impossible, please add a comment explaining why — e.g., "each iteration either updates, creates, or raises, so the loop cannot spin." Right now there's no such justification, and the loop looks like it could run forever.

Everything else looks good:

  • instance is not None correctly handles falsy model instances.
  • The UniqueNameRequired model for SQL Server is a nice detail.
  • CI is green across all backends and CodSpeed shows no performance impact.

I'm happy to approve once the retry loop is bounded (or justified with a comment). If you'd rather keep while True, at minimum add a retry counter and a log line so we can diagnose abnormal spins in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

update_or_create loses defaults under concurrent access

3 participants