fix: restore the package import and reconcile the merged backend behaviour - #66
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
srpatcha
left a comment
There was a problem hiding this comment.
Verified — this is the fix master needs
Reproduced the breakage on origin/master first, so this is not a stale CI badge:
$ ebuild new hi && cd hi && ebuild build
File ".../ebuild/build/dispatch.py", line 133
else:
^^^^
SyntaxError: invalid syntax
$ pytest
2 errors during collection
With this branch merged onto current master: 206 passed, and ebuild build reaches the compiler instead of raising a traceback.
I had independently written the same repair (#70) before seeing this. Yours came first and is the one I would take; I will close mine. Checking them against each other, this branch covers three of the four defects I found:
Two consecutive else: in configure() |
fixed |
"ninja" listed among backends needing no configure — so configure("ninja") silently succeeded |
fixed |
NinjaBackend._object_path called twice, defined nowhere |
fixed |
ninja invoked as python -m ninja |
still present |
The one left
ebuild/build/dispatch.py:244 [sys.executable, "-m", "ninja", "-C", ...]
ebuild/cli/commands.py:690 [sys.executable, "-m", "ninja", "-f", ...]commands.py:690 is the build path, so this is the more interesting of the two. It works today only because pyproject.toml declares ninja>=1.11, which puts the PyPI wheel in the environment. But it means a machine with a perfectly good ninja on PATH is ignored, and the moment that wheel is absent — a system-package install, a distro build, a container that pip-installed with --no-deps — the build fails with No module named ninja rather than anything a developer can act on.
A three-line helper covers it:
def ninja_command() -> list:
exe = shutil.which("ninja")
return [exe] if exe else [sys.executable, "-m", "ninja"]Happy for that to be a follow-up rather than growing this PR — the SyntaxError is the urgent half and I do not want to hold it. Say the word and I will open it against your branch or after it lands.
Verification
Merged onto origin/master locally; pytest 206 passed; ebuild new + ebuild build walked by hand from a clean project.
Verified merge order for the ebuild backlog
#71, #72 and #74 stack in that order; the last of them brings the suite to 316 when applied on top of the merged base. #66 goes first, and supersedes my #70Three PRs fix the same What I rebased#64, #70, #71, #72 and #74 all conflicted in None of #71, #72 or #74 actually touches That rebase also fixed something worth flagging. The footprint commit had normalised 1,814 CRLF line endings in The real change is 89 lines. Still to resolve#65 and #75 conflict with #66 in #64 conflicts more deeply and needs its own pass. VerificationEvery row above is |
The rebase resolution spliced test() into the middle of clean(), leaving
clean()'s body orphaned under a second `def clean` and dropping the
module-level TestOutcome dataclass with its two parser helpers entirely.
The result imported far enough to collect and then failed:
ImportError: cannot import name 'TestOutcome' from 'ebuild.build.dispatch'
and before that, six of master's own clean() tests failed with
NameError: name 'dry_run' is not defined
because the shadowing duplicate did not take the keyword.
This is the same defect this session has been repairing elsewhere — two
versions of one function spliced by a merge that nothing recompiled — and
it is worth naming rather than quietly fixing, because I introduced it by
resolving a conflict with "keep both sides" without rebuilding afterwards.
That is exactly how eos's sync.c ended up referencing fields no version of
the file ever defined at once.
clean() and test() are now separate again and the dataclass is back.
Verified against the branch's base rather than a stale master:
base (#66) 205 passed, 1 skipped
this branch 240 passed, 1 skipped
new failures none
ebuild setup / new / configure / build / test / flash / monitor all present
ebuild test on a real CMake project -> 1 passed, 0 failed exit 0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@srpatcha suggested I flag this here. I hit the same Nothing needed from you, just noting it for the changelog mention they offered. |
Rebuilt on current master. Master has since fixed the dispatch.py SyntaxError and restored NinjaBackend._object_path, so embeddedos-org#66's versions of those are dropped in favour of what landed — including its choice of RuntimeError for an unknown backend, which is now consistent across both dispatch test suites. What follows is what master still does not have. **build.ninja is invalid on Windows.** Ninja splits build statements on unescaped spaces and colons, so a Windows absolute path puts a drive-letter colon where Ninja expects the separator between outputs and the rule name: ninja: error: build.ninja:20: expected build command name build C:\...\main.o: cc main.c ^ near here Every generated file was rejected before a command ran; a POSIX path containing a space fails identically. _ninja_path() escapes `$`, `:` and ` `, applied to build-statement paths only — variable values (cflags, ldflags) are read to end of line and are left alone, since escaping them hands the compiler mangled flags. Four regression tests, one asserting each build statement contains exactly one unescaped colon. **test_shared_library_uses_shared_link_rule fails on macOS.** It asserts the literal "-shared", but _shared_flag() correctly returns "-dynamiclib" on darwin. The implementation is right and the test was not; made it platform-aware. This is the one test failing on master today. **The Windows leg of the test matrix has never run.** `Run test suite` uses backslash line continuations, which PowerShell rejects: ParserError: Missing expression after unary operator '--'. Marked `shell: bash`, which GitHub provides on Windows runners. **macos-13 is a retired runner image**, so those jobs are never assigned a runner and sit queued until they time out. Every other workflow here already uses macos-latest. **`mypy .` checks nothing.** It aborts with `Duplicate module named "tests"` because layers/eosuite/ vendors its own tests/ package. The step is continue-on-error, so this went unnoticed. Excluding layers/ makes it check 84 files; it stays continue-on-error, so the 12 pre-existing findings are visible without gating the build. **ci.yml has no concurrency group**, alone among this repo's workflows, so pushes pile up queued runs competing for the same scarce runners. Also gitignored _build/, which the suite leaves in the repo root. Verified: pytest 292 passed (287 + the fixed shared-flag test + 4 new). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
14928ab to
31ffd98
Compare
|
@srpatcha — thank you for the merge-order analysis, and for marking #70 superseded rather than competing. Since then Dropped (master's versions are in, and are the right calls):
What is left is what master still does not have — now 4 files instead of the previous sprawl:
Re: the CRLF finding in |
With `shell: bash` the Windows leg of the matrix runs pytest for the first
time, and it fails four tests. Neither cause is new; both were simply never
executed.
`test_test_target_links_like_an_executable` picks the link edge out of
build.ninja with `l.split(":")[0]`, meaning "the text before the rule
separator". On Windows the first colon is the drive letter, so that expression
returns "build C" for every line, the `.o` filter never matches, and the test
selects the compile edge and asserts `": link "` against it.
Splitting on the first *unescaped* colon is what was meant, and is now
unambiguous: `_ninja_path()` writes the drive colon as `$:` and leaves exactly
one bare colon per statement, the separator.
build C$:\...\obj\t_smoke\t.c.o: cc t.c -> outputs end in .o (compile)
build C$:\...\t_smoke.exe: link ... -> outputs do not (link)
The three `test_integration_initramfs_security.py` cases fail with WinError 2:
`_create_initramfs()` drives find(1) and cpio(1) directly and neither exists on
a stock Windows runner, so they die before reaching the command-injection
behaviour they exist to check. Building a Linux initramfs is not a Windows
operation, so they skip when the tools are absent — the same shape as the
existing skip in test_ninja_backend.py when no host C compiler is present.
Verified: pytest 292 passed locally; the Windows selection logic checked
against an escaped drive-letter path directly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The architectural problem
import ebuild.build.dispatchraisesSyntaxErroronmaster. The CLI does not start, and five test modules fail at collection.Same root cause as the breakage in the sibling repos: overlapping PRs squash-merged on stale bases, with
masternever re-verified afterwards. Every one of the PRs involved was green on its own branch. What landed is a build tool that cannot be imported, and — once it can be — a default build backend that crashes.What is broken
dispatch.pydoes not parseconfigure()ends with two consecutiveelse:blocks from an unresolved conflict.The two blocks also disagree about the exception type, and each has a test suite behind it:
tests/ebuild/test_dispatch.pyexpectsValueError("Unknown build backend '<name>'")tests/unit/test_dispatch.pyexpectsRuntimeErrormatching"ninja"Both readings are legitimate. An unrecognized backend name is a bad argument; a
"ninja"that reaches the dispatcher is a CLI routing failure, since ebuild's own ninja backend is invoked directly and never dispatched here. Rather than pick a winner and delete someone's regression test, this addsUnknownBackendError(ValueError, RuntimeError)with a message that covers both cases, raised fromconfigure(),build()andclean()."ninja"is also no longer a silent no-op inconfigure(). That silence is what letebuild buildreport "Build completed successfully" without ever running a compiler — the exact failuretests/unit/test_dispatch.pywas written to prevent.The default build backend crashes
NinjaBackend._object_path()was deleted by a cflags refactor while both of its callers survived, sogenerate()dies withAttributeError. Restored, with its target-namespaced object paths — the thing that stops two targets sharing a source file from claiming the same output and making ninja reject the graph withmultiple rules generate ....This also un-breaks the depfile tests from #48: header edits currently leave stale objects behind and the build silently reports success.
Two incompatible shared-library designs both merged
One PR added a dedicated
link_sharedninja rule; another put the platform's shared-object flag intoldflagson the genericlinkrule. Both landed, with a test each, and the tests contradict — one asserts: link_shared, the other asserts: linkplus no-sharedanywhere in the file.The
link_sharedrule was dead: nothing emitted a build line using it, and it hardcoded-shared, which is wrong on macOS (-dynamiclib) and skipped the-L/-lwiring. Dropped it, and rewrote the test that asserted the dead rule to cover the surviving, platform-correct behaviour.The Windows matrix has never run — and the backend has never worked there
The
testmatrix includeswindows-2022, where the default shell is PowerShell. TheRun test suitestep uses backslash line continuations, which PowerShell rejects:So the Windows jobs failed before pytest started, on
masterand on every branch. Markedshell: bash.With those jobs actually running, they exposed a real portability defect:
NinjaBackendnever escaped paths in build statements.Ninja splits build statements on unescaped spaces and colons, so a Windows drive-letter colon lands where Ninja expects the separator between outputs and the rule name. Every generated
build.ninjawas rejected before a command ran; a POSIX path containing a space fails identically._ninja_path()escapes$,:and, applied to build-statement paths only — variable values (cflags,ldflags) are read to end of line and are deliberately left alone, since escaping them would hand the compiler mangled flags. Four regression tests added, including one asserting each build statement contains exactly one unescaped colon.Also on Windows:
tests/ebuild/test_integration_initramfs_security.pydied withWinError 2before reaching the injection behaviour it exists to check —_create_initramfs()drivesfind(1)andcpio(1)directly, neither of which exists there. Building a Linux initramfs is not a Windows operation, so those three now skip when the tools are absent, matching howtest_ninja_backend.pyalready skips without a host C compiler.Finally,
mypy .aborted immediately withDuplicate module named "tests"(layers/eosuite/vendors its owntests/package). Because the step iscontinue-on-error, this went unnoticed and the type check had been checking zero files. Excludinglayers/makes it check 81 source files; it stayscontinue-on-error, so the 11 pre-existing findings are visible without gating the build.Verification
pytest tests/pytest tests/performance/Before this change:
SyntaxErrorat collection, then 11 failures once patched past it, then 4 more on Windows once that leg could run.Relationship to open PRs
#65 also fixes the
dispatch.pyelse, and #64 also restores_object_path. Neither reconciles the two contradictory dispatch test suites — they pick one exception type, which leaves the other suite failing — and neither addresses the duplicate shared-library design or the Windows breakage. Happy to rebase onto whichever lands first.🤖 Generated with Claude Code