Fix StatefulSet recreate failure handling

[realyota]

Fix for #1995

Test was written by Codex.

Important items to consider before making a Pull Request

Please check items PR complies to:

• [ X] All commits in the PR are squashed. More info
• [X ] The PR is made into dedicated next-release branch, not into master branch¹. More info
• [ X] The PR is signed. More info

--

¹ If you feel your PR does not affect any Go-code or any testable functionality (for example, PR contains docs only or supplementary materials), PR can be made into master branch, but it has to be confirmed by project's maintainer.

[realyota]

Interesting, same problem fixed here: #1993 and it is "treating the cause". Both PRs do not fix all problems. #1997 it is more defensive (catches a wider family of failures, not just 409 on scale-down) but it is missing doDeleteStatefulSet early-return when scale-to-0 (Update) fails and Delete never happens. Probably best way will be to merge #1993 and then rebase on #1997

[alex-zaitsev]

Short answer: the delete-path part of #1997 is on a real, used path. The create-path Abort rewrite is weaker, and much of the PR is already superseded on current 0.27.3.

What the PR changes

1. recreateStatefulSet — stop ignoring doDeleteStatefulSet errors
2. doDeleteStatefulSet — return delete/get failures (and nil on NotFound)
3. doCreateStatefulSet — create API failure → ErrCRUDAbort (was ErrCRUDRecreate)
4. shouldAbortOrContinueCreateStatefulSet — ErrCRUDRecreate → Abort (was swallow as success)

Is it actually used?

Delete/recreate path — yes, real.

recreateStatefulSet is called from:

• ForceRecreate
• updateStatefulSet when update fails / STS not ready → falls back to recreate

That is the #1995 bug: scale-down/delete fails, create is still attempted / error ignored, reconcile looks successful while STS/pod are missing. Honoring delete failure is necessary and reachable.

Create-path Abort — mixed.

After #1997 itself changes doCreateStatefulSet to return ErrCRUDAbort on Create failure:

• the defensive ErrCRUDRecreate → Abort branch becomes almost dead for Create-API failures
• wait-for-launch failures go through OnStatefulSetCreateFailed, which returns Abort / Ignore, never ErrCRUDRecreate

So item (4) is mostly defensive dead code once (3) lands. Item (3) itself is used (Create can fail), but Abort vs Recreate is a policy choice.

Already superseded locally

Your current tree already has the useful delete fixes (and more), largely from #1993 plus follow-ups:

• recreateStatefulSet already checks delete error and aborts
• doDeleteStatefulSet already returns errors and does best-effort scale-to-0

And for create failures, current code deliberately keeps ErrCRUDRecreate and propagates it (see TestCreateStatefulSet_AlreadyExistsPropagatesAsRecreate) — the opposite of #1997’s Abort mapping — so AlreadyExists / stale-informer races retry instead of hard-abort.

Verdict

Piece	Really used?	Still needed on current tree?
Honor delete failure in recreate	Yes	Mostly already present
Return delete errors from doDelete	Yes	Mostly already present (#1993+)
Create failure → Abort	Reachable, but policy	Superseded by Recreate-propagate
Recreate→Abort in create completion	Barely, after PR’s own change	Conflicts with current tests

So: the bugfix idea is real and the recreate/delete path matters, but as a PR against current 0.27.3, #1997 is largely obsolete / overlapping with #1993, and its create→Abort half is not what the tree wants anymore.