fix(windows): survive cmd.exe's 8191 character command line limit#1209
Conversation
- with big extension stacks (imagick) the link command lines exceed what cmd.exe allows and fail with 'The command line is too long' - pass the collected LIBS_CLI/LIBS_CGI/LIBS_MICRO lib lists to link.exe through @response files, like php-src already does for its object lists - compile the frankenphp -mthreads shim instead of generating a .bat: batch files run through cmd.exe and cap cgo link lines at 8191 chars, a real executable gets the full 32K and past that go writes a response file by itself; still probe-gated, still a stopgap for golang/go#80290
|
I haven't tried it yet. BTW if this also solves the long path build problem for ImageMagick, then it's best not to put ImageMagick in C:\im, otherwise you won't be able to reset the build using bin/spc reset. |
- C:\im was outside the project so bin/spc reset could not clean it - with the command line limit fixed a short base path is no longer needed; keep SPC_IMAGEMAGICK_BUILD_DIR as an override
|
Silly question, but can't we just use powershell? |
That's a good question, but I suspect we shouldn't switch to PowerShell, and seems no more benefits. Looks like PowerShell would only address the cgo wrapper. It would not solve the separate oversized nmake LIBS_* command. Also PowerShell is too slow. It takes 2 seconds to start on my vm for the first time. |
|
But with AI, I wouldn't mind giving it a try, though it doesn't seem to make much sense. |
PowerShell has no 8k character limit |
|
@crazywhalecc good point, tested it: with the command line fix the short path isn't needed anymore, both imagick combos build fine with the tree under @henderkes I looked into it, powershell would work, it has no 8k limit. The problem is that cgo calls CC a few hundred times per build, and every call would start a new powershell, which takes 150-500ms each time: Measure-Command { powershell -NoProfile -Command "" } | Select TotalMillisecondsThat adds a minute or more to every build. The compiled shim starts instantly. It also passes the command line through untouched, while powershell likes to re-quote arguments on its way to clang. |
|
No... I mean literally never using cmd at all and instead using powershell. |
|
Oh ok. What about just calling CreateProcess directly everywhere instead of cmd/pwsh? cmd would only stay for the .bat scripts (configure.bat, vcvars). |
|
I wouldn't very mind switching to PowerShell entirely, but switching to PowerShell might present us with many new problems, whereas currently cmd only has the issue of length. Given that all our current build scripts' escaping, environment variable passing, quotes, and dependent vcvars batch files are written around cmd, switching to PowerShell will still call cmd unless we completely refactor. Moreover, even if we bypass cmd/pwsh and directly use proc_open(bypass_shell: true), I'm not sure if it will still match our current terminal input/output and logging functionality. |
This fixes the
imagick [frankenphp]failure from #1207 (The command line is too long, run). Moving ImageMagick toC:\imcouldn't help: the length comes from the lib list and the objects, not the base path.Two things going on:
-mthreadswrapper was a .bat, and batch files can only run through cmd.exe, which caps the command line at 8191 chars. A real executable gets ~32K, and past that go writes a response file by itself. So the wrapper is now a small C shim, generated and compiled with the already-detected clang at build time. Same probe, same stripping, still a stopgap for cmd/go: cgo passes -mthreads to the C compiler on Windows, rejected by clang targeting MSVC golang/go#80290. I know you asked for a script instead of a compiled wrapper back in fix(frankenphp): strip Go's -mthreads so the Windows build survives Clang >= 20 #1201, but the script turned out to be exactly the 8k ceiling 😅@responsefiles, same thing php-src already does for its object lists.Verified on windows-latest with the failing combo (
imagick --build-cli --build-frankenphp --enable-zts) plus imagick cli/cgi/micro and pdo_pgsql,pgsql: https://github.com/m-this/static-php-cli/actions/runs/29072901492