Fix linker callback leak for invalid UTF-8 names

[grandpig]

Summary

Validate linker function module and name strings before storing their Go callbacks.

Linker.FuncNew and Linker.FuncWrap previously inserted their callbacks into the global callback maps before calling wasmtime_linker_define_func.

In Wasmtime v47, wasmtime_linker_define_func validates the module and name as UTF-8 before constructing the Rust callback closure:

let module = to_str!(module, module_len);
let name = to_str!(name, name_len);
let cb = c_callback_to_rust_fn(callback, data, finalizer);

If either string contains invalid UTF-8, the C API returns before it takes ownership of data. Consequently, the Go finalizer is never called and the entry remains in gEngineFuncNew or gEngineFuncWrap, retaining the callback and everything captured by it.

This change:

• rejects invalid UTF-8 module and function names before inserting callbacks;
• resolves the linker and function type pointers before callback insertion, so panics caused by closed objects cannot leave callback entries behind;
• adds regression tests for invalid module names, invalid function names, and closed linkers.

Reproduction

err := linker.FuncWrap(
    string([]byte{0xff}),
    "host",
    func() {},
)

Before this change, every call returned an error but added a permanent entry to the global callback map.

[alexcrichton]

Thanks for the PR, but I think the best fix here would be to fix the C API of Wasmtime itself. Would you be up for sending a PR there to run the destructors in the case that these fail? For example moving the closure construction earlier

[grandpig]

Thanks for the PR, but I think the best fix here would be to fix the C API of Wasmtime itself. Would you be up for sending a PR there to run the destructors in the case that these fail? For example moving the closure construction earlier

Thanks for the suggestion. I sent the fix to the Wasmtime C API here: bytecodealliance/wasmtime#14126

It moves callback closure construction before UTF-8 name parsing for the checked, unchecked, and async linker function APIs, with regression tests verifying that the finalizers run when parsing fails.

[alexcrichton]

Thanks! I'm going to close this since I think that's the best fix and that'll get picked up here on the next release