fix(integration): build initramfs without host cpio - #90
Conversation
srpatcha
left a comment
There was a problem hiding this comment.
Approving. I validated the archive output independently rather than trusting the
unit tests, because a hand-written newc writer is exactly the kind of thing
that passes its own tests and produces something a real tool will not read.
Independent validation
Built an initramfs from a rootfs with nested directories, a shell script, an
ELF-magic binary and a text payload, then extracted it with GNU cpio:
$ gzip -dc initramfs.cpio.gz | cpio -idmv
.
./bin
./bin/prog
./etc
./etc/motd
./init
2 blocks
$ cat etc/motd
hello payload
-rwxrwxr-x init
-rwxrwxr-x bin/prog
Every path present, payload byte-identical, execute bits preserved on both the
script and the binary. The four-byte alignment and header fields are right —
cpio is strict about both and would have complained.
Confirmed the host dependency is genuinely gone: _create_initramfs() contains
no subprocess or shutil.which call, only the docstring describing the
find | cpio | gzip pipeline it replaced.
full suite: 290 passed, 1 skipped
On the Windows mode handling
def _newc_mode(path: Path, metadata) -> int:
"""Windows reports regular files as non-executable even when they are
shell scripts or cross-compiled ELF binaries..."""Right call, and right to be conservative about it. Sniffing content to restore
execute bits only on Windows, leaving POSIX modes untouched, means the common
case carries no heuristic at all. An initramfs whose /init is not executable
is a kernel panic at boot with a famously unhelpful message, so getting this
wrong is expensive and getting it approximately right on the platform that
cannot express the information is the best available answer.
On the evidence row
QEMU boot was not run on Windows.
1 unrelated failure in the pre-existing Windows Ninja path assertion,
recorded as T-002.
Stating what was not verified, and separating a pre-existing failure from
anything this PR caused, is more useful than an unqualified pass. It let me spend
verification effort on the archive format instead of re-running your suite.
For what it is worth: I could not run the QEMU boot either — QEMU is not
installed in this environment, which is also why nothing in this organisation has
yet been observed to boot. Extraction by an independent implementation is the
strongest check available short of that, and it passes.
One thought, not blocking
_newc_padding() and the record writer are general enough to be worth testing
against a fixture archive produced by real cpio, byte for byte, if this format
grows more cases (device nodes, long paths over 256 bytes). The current tests
cover behaviour; a golden-bytes comparison would pin the wire format the way
eBoot#66 pins the .efw header.
`import ebuild.cli.commands` raises NameError on master, so the `ebuild` command does not start and pytest cannot collect nine test modules -- the suite reports "9 errors during collection" and runs zero tests. Four PRs landed their tests while their implementation was discarded during conflict resolution, and the merges that did it were not gated by a run: f5209ac, current master, has no CI runs at all. #64 (6a22e36) tests/unit/test_golden_path_commands.py landed; _workspace_repo_paths, _cached_repo_libraries, _record_board_selection, _EOS_PROJECT_CONFIG and the `configure --board` option did not. #89 (32348f0) commands.py gained five `re.` uses, no `import re`. #90 (d3958f7) the initramfs test kept its assertions but lost its `_newc_members` helper and its `os` / `stat` imports. #92 registry.version_key was renamed version_sort_key without updating its caller in the tests. Restoring them, taken verbatim from the branches that were merged: * commands.py: `import re`; `_workspace_repo_paths`, `_cached_repo_libraries`, `_record_board_selection`, `_EOS_PROJECT_CONFIG`; the `--board` option on `configure` and the call that persists it; and the `{**_workspace_repo_paths(), **...}` merge at both `_install_packages` call sites. Without that last hunk a scaffolded project that `use`s eos or eboot gets no include or link paths from ~/.ebuild/repos, which is the "fatal error: eos/hal.h: No such file or directory" that #64 fixed. Without the option, the documented golden path -- `configure --board` then a bare `build` -- fails with "no such option: --board". * the initramfs test: `os` / `stat` imports and the `_newc_members` newc parser the assertions call. * test_package_registry.py: import the name the module now exports. Collection goes from 9 errors / 0 tests to 446 passing. The 12 that still fail are pre-existing and out of scope here: `doctor` (#72) and `package` (#77) are two more features whose implementations the same merges dropped and which should be restored by their authors, and the rest are separate bugs -- see the PR description. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Integration builds currently create
initramfs.cpio.gzby launching hostfind,cpio, andgzipprocesses. That toolchain is absent on Windows andmakes a core build artifact depend on Unix host utilities.
This change replaces the subprocess pipeline with a standard-library
newcserializer, so integration builds create the initramfs on every supported host.
Type of Change
Changes
newcrecords directly, including requiredalignment and the
TRAILER!!!marker.identity without duplicating hard-linked payloads.
host mode bits do not represent Linux executability.
cpiorequirement from theQEMU error message.
cpiotest assertions with an internal format parser and addcoverage for literal metacharacter paths, mode bits, symlinks, and hard links.
The record layout and hard-link behavior follow the Linux kernel's
documented initramfs buffer format.
Testing
5 passed, 1 skippedbsdtarextraction validated both hard-link names, sharedinode identity, and payload content
flake8 --select=Fpasses for both changed Python filesFull Python suite:
288 passed, 2 skipped, 1 failed. The single failure is anexisting Windows-only Ninja path-parsing assertion in
test_test_target_links_like_an_executable; it is unrelated to initramfs codeand is recorded as T-002 in
TASKS.md.Pre-Submission Checklist
<type>(<scope>): <description>conventionmasterRelated Issues
None.
Screenshots / Logs
Not applicable.
Additional Notes
explicit error; the generated project rootfs currently contains regular
files, directories, and symlinks.
are not reliably available; it runs on POSIX CI hosts.