Add HL_DEBUG_CODEGEN_LOG_FILE to redirect debug() logs

[alexreinking]

Applies globally, resolved at first debug() invocation. Maintains its own buffer so that it can send a whole chain of << outputs to the OS file descriptor in one atomic write. As long as the output buffer isn't exhausted, it is robust to parallel appends to files.

Remaps /dev/stdout to std::cout and /dev/stderr to std::cerr on all platforms (including Windows) both for compatibility and for less surprise when mixing std::cerr and debug(0).

Checklist

• Tests added or updated (not required for docs, CI config, or typo fixes)
• Documentation updated (if public API changed)
• Python bindings updated (if public API changed)
• Benchmarks are included here if the change is intended to affect performance.
• Commits include AI attribution where applicable (see Code of Conduct)

[alexreinking]

Funny. I wrote this with Claude, but I guess I steered it so much, it didn't want to credit itself 😂

[codecov]

Codecov Report

❌ Patch coverage is 75.55556% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.95%. Comparing base (e627fbe) to head (ad83655).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
src/Debug.cpp	72.50%	7 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9361      +/-   ##
==========================================
- Coverage   69.97%   69.95%   -0.03%     
==========================================
  Files         259      259              
  Lines       79158    79201      +43     
  Branches    19293    19301       +8     
==========================================
+ Hits        55394    55407      +13     
- Misses      17898    17931      +33     
+ Partials     5866     5863       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mcourteaux]

Curious why you want this. I'm afraid this is limitedly useful when debugging tests: the tests use regular printf() almost everywhere, which are now not redirected.

When I first saw the title of this PR, I guessed you were increasing the log level for the log file by one over the console log. But it's equal it seems.

[alexreinking]

Curious why you want this. I'm afraid this is limitedly useful when debugging tests: the tests use regular printf() almost everywhere, which are now not redirected.

It's not for debugging tests. It's for collecting data. We have hundreds to thousands of generator configurations and we want to get internal compiler debug prints out of them. Take for example,

Halide/src/Simplify.cpp

Lines 448 to 503 in e627fbe

	debug(1) << [&]() -> std::string {
	struct RenameVariables : public IRMutator {
	using IRMutator::visit;
	Expr visit(const Variable *op) override {
	auto it = vars.find(op->name);
	if (const std::string *n = lets.find(op->name)) {
	return Variable::make(op->type, *n);
	} else if (it == vars.end()) {
	std::string name = "v" + std::to_string(count++);
	vars[op->name] = name;
	out_vars.emplace_back(op->type, name);
	return Variable::make(op->type, name);
	} else {
	return Variable::make(op->type, it->second);
	}
	}
	Expr visit(const Let *op) override {
	std::string name = "v" + std::to_string(count++);
	ScopedBinding<string> bind(lets, op->name, name);
	return Let::make(name, mutate(op->value), mutate(op->body));
	}
	int count = 0;
	map<string, string> vars;
	Scope<string> lets;
	std::vector<pair<Type, string>> out_vars;
	} renamer;
	Expr renamed = renamer(e);
	// Look for a concrete counter-example with random probing
	static std::mt19937 rng(0);
	for (int i = 0; i < 100; i++) {
	map<string, Expr> s;
	for (const auto &p : renamer.out_vars) {
	if (p.first.is_handle()) {
	// This aint gonna work
	return "";
	}
	s[p.second] = make_const(p.first, (int)(rng() & 0xffff) - 0x7fff);
	}
	Expr probe = unwrap_tags(simplify(substitute(s, renamed)));
	if (!is_const_one(probe)) {
	// Found a counter-example, or something that fails to fold
	return "";
	}
	}
	ostringstream ss;
	ss << "Failed to prove, but could not find a counter-example:\n " << renamed << "\n"
	<< "Original expression:\n"
	<< orig << "\n";
	return ss.str();
	}();

If I want to get all of the failed proofs out of a long build process, I can just write:

$ export HL_DEBUG_CODEGEN="1,Simplify.cpp:448"
$ export HL_DEBUG_CODEGEN_LOG_FILE="$PWD/failed-proofs.log"
$ ./build-script.sh

Where build-script.sh is a long, noisy, build process that produces megabytes of irrelevant output. In our case, build-script.sh prefixes lines with [timestamp] [pid] ... so even a very accurate grep needs more processing to clean up. Also, not grepping means I can watch the build process like normal.