fix(workflows): reject a condition that has no {{ }} block - #4182
fix(workflows): reject a condition that has no {{ }} block#4182ntdatt812 wants to merge 3 commits into
Conversation
`evaluate_condition` resolves its argument through `evaluate_expression`,
which only substitutes `{{ ... }}` blocks. A string with no such block
comes back unchanged and — unless it reads `true`/`false` — is then
coerced by `bool()`. So a condition authored without the braces is never
evaluated at all:
evaluate_condition("inputs.count > 100", ctx) -> True
evaluate_condition("{{ inputs.count > 100 }}", ctx) -> False
with `inputs.count == 5` in both cases. An `if` step always takes `then`,
and a `while`/`do-while` step always runs to `max_iterations` — ten agent
invocations for a loop the author expected to stop.
This is the same silent-truthiness authoring mistake the three step
validators already reject for a list/dict/number condition, and it is
easier to make: GitHub Actions accepts a bare expression in `if:`, so the
brace-less form is a habit to bring here.
Adds `condition_is_never_evaluated()` and wires it into the `if`,
`while` and `do-while` validators, so the mistake surfaces at validation
with the corrected form spelled out. Boolean literals, real bools, empty
strings and any string containing `{{` stay valid — runtime behaviour is
unchanged.
There was a problem hiding this comment.
Pull request overview
Adds validation to reject brace-less workflow conditions that would otherwise always evaluate as true.
Changes:
- Adds a shared condition-validation helper.
- Integrates validation into
if,while, anddo-whilesteps. - Adds runtime and validator regression tests.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/workflows/expressions.py |
Adds condition detection helper. |
src/specify_cli/workflows/steps/if_then/__init__.py |
Validates if conditions. |
src/specify_cli/workflows/steps/while_loop/__init__.py |
Validates while conditions. |
src/specify_cli/workflows/steps/do_while/__init__.py |
Validates do-while conditions. |
tests/unit/test_condition_expression_block.py |
Adds regression coverage. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 5/5 changed files
- Comments generated: 4
- Review effort level: Balanced
| stripped = condition.strip() | ||
| if not stripped or stripped.lower() in ("true", "false"): | ||
| return False | ||
| return "{{" not in stripped |
| f"If step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
| f"While step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
| f"Do-while step {config.get('id', '?')!r}: 'condition' " | ||
| f"{config['condition']!r} has no '{{{{ }}}}' block, so it is never " | ||
| "evaluated and is always true. Wrap the expression: " | ||
| '"{{ ' + str(config["condition"]).strip() + ' }}".' |
Two gaps in the condition validator, both raised in review.
An opening `{{` with no `}}` after it is never substituted either:
_interpolate_expressions takes its `raw_close == -1` branch and appends
the tail verbatim. So `condition: "{{ inputs.count > 100"` -- and the
reversed `"}} inputs.count > 100 {{"`, whose only `{{` is last -- come
back unchanged and are coerced to true exactly like a brace-less string.
The helper now looks for a complete block rather than an opening one.
The suggested correction was interpolated into a double-quoted scalar,
so a condition containing a double quote produced YAML that does not
parse: `condition: "{{ inputs.name == "zzz" }}"` raises a ParserError.
format_condition_correction() now picks the quoting from the content and
drops a stray delimiter instead of nesting a second one, so the message
stays paste-ready. All three validators share it.
Tests: 30 more cases -- the incomplete forms, and a YAML round trip over
conditions holding single quotes, double quotes, both, and backslashes,
asserting each correction loads back exactly and is not re-flagged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Both review points were correct — thanks. Fixed in c0c291c. 1. Unterminated / reversed delimiters slipped through Confirmed against the interpolator rather than assumed.
The helper now requires a complete block: it finds the first 2. The correction was not valid YAML Reproduced exactly as described — Added
Tests — the file goes from 40 to 70 cases. The new ones cover the incomplete forms, and run every correction through
|
| # An opening ``{{`` with no ``}}`` anywhere after it is never substituted | ||
| # either: ``_interpolate_expressions`` takes its ``raw_close == -1`` branch | ||
| # and appends the tail verbatim. So ``{{ inputs.count > 100`` -- and the | ||
| # reversed ``}} inputs.count > 100 {{``, whose only ``{{`` is last -- come | ||
| # back unchanged and are just as silently true as a brace-less string. | ||
| return stripped.find("}}", open_at + 2) == -1 |
| if '"' not in wrapped and "\\" not in wrapped: | ||
| return '"' + wrapped + '"' | ||
| if "'" not in wrapped: | ||
| return "'" + wrapped + "'" | ||
| return '"' + wrapped.replace("\\", "\\\\").replace('"', '\\"') + '"' |
mnriem
left a comment
There was a problem hiding this comment.
Please address Copilot feedback
…h json.dumps
Both follow-up review points were right.
The completeness check used a plain `find("}}")`, but the substituter closes a
block with a quote-aware scan. So `condition: "{{ inputs.x == '}}'"` looked
complete to the validator while `_interpolate_expressions` found no close, fell
to its raw-close branch, evaluated a truncated body and left residual text
(`False'`) -- a non-empty string, hence true. Rather than restate the quote
rules a third time, the scan moves out of `_interpolate_expressions` into
`_find_block_close`, which the validator now calls: the check and the
substitution it predicts can no longer disagree. A `}}` that is genuinely
inside a string argument still does not close early, so
`{{ inputs.text | default('}}') }}` and `{{ inputs.x == '}}' }}` stay accepted.
The correction's quoting enumerated the characters it escaped, and the
enumeration was short: a condition loaded from a YAML literal block can carry a
newline, which a double-quoted scalar folds, so the corrected form did not
round-trip. `json.dumps` decides it instead -- every JSON string is a valid
YAML double-quoted scalar and it escapes quotes, backslashes, newlines and the
other control characters. `ensure_ascii=False` keeps a non-ASCII operand
readable rather than expanding it into numeric escapes.
Tests: 70 -> 83. The quoted-delimiter condition joins the incomplete-block set,
and the round-trip set gains multiline, newline-with-quote, tab, carriage
return and non-ASCII operands. All four new cases fail on the previous commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@mnriem — both follow-up findings addressed in 1. The Confirmed exactly as described:
Rather than restate the quote rules a third time in this module, I lifted the scan out of 2. Hand-rolled quoting missed control characters Also reproduced — a condition carrying a newline did not round-trip:
Switched to Tests 70 → 83. The quoted-delimiter condition joins the incomplete-block set; the round-trip set gains multiline, newline-with-quote, tab, carriage return and a non-ASCII operand. All four new cases fail on the previous commit ( Verification on this head: |
Defect
evaluate_conditionresolves its argument throughevaluate_expression, which only substitutes{{ ... }}blocks. A string with no such block comes back unchanged, and — unless it readstrue/false— is then coerced bybool(). So a condition authored without the braces is never evaluated at all.With
inputs.count == 5:Every brace-less condition is true, whatever it says. An
ifstep always takesthen; awhile/do-whilestep never terminates on its condition and runs tomax_iterations— ten agent invocations for a loop the author expected to stop after one.Nothing reports it. The workflow validates, runs, and takes the wrong branch silently.
Why this is worth a validation error
The three step validators already reject a list/dict/number condition, and the comment there states the reason exactly:
A brace-less string is the same failure mode, and a likelier mistake: GitHub Actions accepts a bare expression in
if:(if: github.event_name == 'push'), so an author arriving from Actions writes the brace-less form by habit — and unlike[1, 2],condition: inputs.count > 100looks completely correct on the page.Fix
condition_is_never_evaluated()inexpressions.py, wired into theif,whileanddo-whilevalidators. The error names the problem and hands back the corrected form:Validation only — no runtime behaviour changes. Still valid, and covered by tests:
"{{ ... }}"in any position,"true"/"false"in any case, realbools, empty and whitespace strings, and every non-string type (already handled by the branch above).Tests
New
tests/unit/test_condition_expression_block.py, 40 cases:FalsevsTrue— so the defect stays documented even if the validator changesThe same 22 pre-existing failures in both runs — all
symlink_toon Windows without the privilege (OSError: [WinError 1314]), unrelated to this change. Everything added is the +40 new tests.