Recover from panics in async capability calls

[vreff]

Capability calls from the wasm host module spin up a goroutine to handle the call, then immediately return. Those goroutines seem to be unhandled by wasmtime if they panic.

[github-actions]

📊 API Diff Results

No changes detected for module github.com/smartcontractkit/chainlink-common

View full report

[Copilot]

🟢 Approval recommended

Only a minor metric-contract nit remains; no blocking issues were identified.

Pull request overview

Adds recovery for panics in asynchronous wasm capability and secrets calls, converting them into reported errors instead of terminating execution.

Changes:

• Recovers and normalizes panics from capability and secrets calls.
• Logs recovered panics and records host metrics.
• Adds regression tests for error propagation, awaiting, and limiter release.

File summaries

File	Summary
pkg/workflows/wasm/host/execution.go	Adds recovery helpers for asynchronous execution.
pkg/workflows/wasm/host/execution_panic_test.go	Tests panic recovery and resource cleanup.

Review details

Suppressed comments (1)

pkg/workflows/wasm/host/execution.go:140

• This records panics from ExecutionHelper goroutines in platform_wasm_host_panic_recovered_total, but that counter is registered as counting recovered “wasmtime-go” panics (module_metrics.go:23-24). Async helper panics are a different source, so dashboards and alerts will misclassify this new failure mode; update the metric contract/description or use a separate counter before adding this increment.

		e.module.metrics.IncHostFnPanicRecovered()

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[cedric-cordenier]

@vreff Wasmtime will already be catching this panic for you, and then bubbling it back up to the call to module.Execute

This gets caught here:

chainlink-common/pkg/workflows/wasm/host/module.go

Line 675 in 1d3a14a

func callStart(m *module, instance *wasmtime.Instance, store *wasmtime.Store) (result any, err error) {

and should stop the panic

If this isn't working for you, I'd love to understand more

[vreff]

@vreff Wasmtime will already be catching this panic for you, and then bubbling it back up to the call to module.Execute

This gets caught here:

chainlink-common/pkg/workflows/wasm/host/module.go

Line 675 in 1d3a14a

func callStart(m *module, instance *wasmtime.Instance, store *wasmtime.Store) (result any, err error) {

and should stop the panic

If this isn't working for you, I'd love to understand more

The issue seems to be that the capability call spins up a new goroutine and then returns nil immediately. What happens if that instance shuts down for some other reason, like a timeout, and then the goroutine it started panics?

Problematic code:

	go func() {
		defer free()

		resp, err := e.executor.CallCapability(ctx, req)

		if err != nil {
			errString := err.Error()

			if caperror, ok := errors.AsType[caperrors.Error](err); ok {
				errString = caperror.SerializeToString()
			}
			resp = &sdkpb.CapabilityResponse{
				Response: &sdkpb.CapabilityResponse_Error{
					Error: errString,
				},
			}
		}

		select {
		case <-ctx.Done():
		case ch <- resp:
		}
	}()

	return nil

This just seems like a new goroutine, which is not guaranteed to be tracked by that wasm instance.

[vreff]

@cedric-cordenier I've reworked this PR as simply a test to show the behavior I'm concerned about. LMK if some aspect of the test is missing context.

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: 67e2f77	Previous: 22ea6bc	Ratio
BenchmarkKeystore_Sign/nop/in-process	836.3 ns/op	232.6 ns/op	3.60

This comment was automatically generated by workflow using github-action-benchmark.