audio: microwakeword: add mww component by singalsu · Pull Request #11135 · thesofproject/sof

singalsu · 2026-08-27T13:30:17Z

No description provided.

Copilot

🟡 Changes recommended

There are correctness issues that will break integration (notably the KPB UUID mismatch with the UUID registry) and should be fixed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces a new microWakeWord (MWW) keyword-spotting component and integrates it into SOF’s IPC4/Zephyr-based build, including new topology2 pipelines (KPB-based Wake-on-Voice branches), rimage manifests, and supporting LLEXT/AMS/DP-scheduler plumbing needed to run TFLite Micro–based C++ extensions.

Changes:

Add the microwakeword audio component (C/C++ + Kconfig/CMake + rimage TOML) and offline training/tuning scripts.
Add topology2 widget/pipeline templates and platform overlays to enable MWW+KPB Wake-on-Voice capture branches on HDA and SDW.
Extend DP-scheduler integration (optional DP→DP binding), AMS payload handling, and LLEXT build/export behavior for C++ multi-TU libraries.

File summaries

File	Description
zephyr/Kconfig	Broadens IPC4 pipeline2.0 + DP scheduler defaults to ACE/CAVS; adds DP→DP bind Kconfig.
zephyr/CMakeLists.txt	Adds `-Wl,-Bsymbolic-functions` for LLEXT shared builds to avoid unresolved local cross-TU calls.
uuid-registry.txt	Registers UUID for new `mww` module.
tools/topology/topology2/sof-hda-generic.conf	Adds optional HDA mic MWW/KPB capture include and required class includes.
tools/topology/topology2/platform/intel/sdw-jack-mww-kpb.conf	New SDW jack MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/sdw-dmic-mww-kpb.conf	New SDW DMIC MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/hda-mic-mww-kpb.conf	New HDA analog MWW/KPB WoV capture branch overlay topology.
tools/topology/topology2/platform/intel/dmic1-mfcc.conf	Removes redundant MFCC include (comment-only change).
tools/topology/topology2/include/pipelines/cavs/src-kpb-be.conf	New reusable SRC→KPB backend pipeline class for WoV branching.
tools/topology/topology2/include/pipelines/cavs/host-gateway-src-mfcc-mww-capture.conf	Adds a detection pipeline template combining SRC+MFCC+MWW.
tools/topology/topology2/include/pipelines/cavs/host-gateway-micsel-mfcc-mww-capture.conf	Adds a stereo-compatible detection pipeline template using micsel+MFCC+MWW.
tools/topology/topology2/include/components/mww.conf	Adds topology2 widget class definition for `mww`.
tools/topology/topology2/include/components/mfcc/mel80.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel80_compress.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel80_compress_dtx.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/mel40.conf	Adds new 40-bin MFCC config blob.
tools/topology/topology2/include/components/mfcc/mel40_compress.conf	Adds new compressed-output 40-bin MFCC config blob.
tools/topology/topology2/include/components/mfcc/mel40_10ms.conf	Adds new 40-bin, 10ms-hop MFCC config blob for MWW.
tools/topology/topology2/include/components/mfcc/mel40_10ms_compress.conf	Adds compressed-output 40-bin, 10ms-hop MFCC config blob.
tools/topology/topology2/include/components/mfcc/default.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/mfcc/ceps13_compress_dtx.conf	Updates exported MFCC config date header.
tools/topology/topology2/include/components/kpb.conf	Updates KPB UUID (currently inconsistent with uuid registry; see comment).
tools/topology/topology2/include/common/common_definitions.conf	Adds feature flags for enabling MWW/KPB overlays.
tools/topology/topology2/include/common/abi.conf	Adds ABI manifest blob definition.
tools/topology/topology2/include/bench/mfccmel40_10ms_s32.conf	Adds MFCC mel40_10ms benchmark include (S32).
tools/topology/topology2/include/bench/mfccmel40_10ms_s24.conf	Adds MFCC mel40_10ms benchmark include (S24).
tools/topology/topology2/include/bench/mfccmel40_10ms_s16.conf	Adds MFCC mel40_10ms benchmark include (S16).
tools/topology/topology2/include/bench/mfcc_controls_playback.conf	Adds bench parameter key for mel40_10ms MFCC blob.
tools/topology/topology2/include/bench/mfcc_controls_capture.conf	Adds bench parameter key for mel40_10ms MFCC blob.
tools/topology/topology2/development/tplg-targets.cmake	Adds development topology targets enabling MWW/KPB branches for HDA + SDW.
tools/topology/topology2/development/tplg-targets-bench.cmake	Adds bench target/params for mel40_10ms MFCC benchmark.
tools/topology/topology2/cavs-sdw.conf	Adds optional includes and keys for SDW MWW/KPB overlays + required class includes.
tools/topology/topology2/cavs-benchmark-hda.conf	Adds benchmark configs for mfccmel40_10ms variants.
tools/rimage/config/wcl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/tgl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/tgl-h.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/ptl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/mtl.toml.h	Includes MWW module TOML when enabled.
tools/rimage/config/lnl.toml.h	Includes MWW module TOML when enabled.
src/platform/intel/cavs/platform.c	Initializes DP scheduler when enabled.
src/library_manager/llext_manager_dram.c	Ensures VMA cleanup on restore failure.
src/lib/cpp_new_export.cpp	Exports C++ allocation/runtime symbols for LLEXT modules.
src/lib/CMakeLists.txt	Builds the new C++ symbol export shim when CONFIG_CPP is enabled.
src/lib/ams.c	Adds inline payload data copy into AMS slots; exports ams_send().
src/ipc/ipc4/helper.c	Adds optional DP→DP binding support and DP ring buffer attachment changes.
src/ipc/ipc4/ams_helpers.c	Exports AMS helper functions for external users/modules.
src/include/sof/lib_manager.h	Extends lib manager module struct with export segment + VMA tracking.
src/include/sof/audio/mfcc/mfcc_vad.h	Tunes MFCC VAD constants (noise rise alpha + threshold).
src/audio/module_adapter/module_adapter.c	Propagates DP domain from extended init; minor sync constant fix.
src/audio/microwakeword/tune/sof_mww_verify.py	Adds streaming verification script for quantized MWW models.
src/audio/microwakeword/tune/sof_mww_train_pipeline.sh	Adds end-to-end dataset→features→train→verify pipeline runner.
src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh	Adds script to prepare silence/unknown classes from Speech Commands v2.
src/audio/microwakeword/tune/sof_mww_plot_mtrace.py	Adds mtrace visualization tool for MWW diagnostics.
src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset.sh	Adds multi-speaker synthetic keyword dataset generator (has duplicated helper; see comment).
src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset_from_dir.sh	Adds real-speech ingestion + augmentation dataset builder.
src/audio/microwakeword/tune/sof_mww_dataset.py	Adds feature loader + augmentation utilities for training.
src/audio/microwakeword/tune/sof_mfcc_extract_features.sh	Adds batch feature extraction via sof-testbench4 + topology2 bench tplg.
src/audio/microwakeword/tune/README.md	Documents offline MWW training/tuning toolchain and workflows.
src/audio/microwakeword/README.md	Documents MWW component architecture, dataflow, and deployment notes.
src/audio/microwakeword/mww.toml	Adds rimage module manifest entry for MWW.
src/audio/microwakeword/mww_model.h	Adds C API surface for the MWW TFLM inference wrapper.
src/audio/microwakeword/mww_model.cc	Implements TFLM interpreter setup + streaming inference wrapper.
src/audio/microwakeword/mww_model_data.h	Adds generated model-data header declaration.
src/audio/microwakeword/llext/llext.toml.h	Adds LLEXT-specific TOML wrapper for MWW module.
src/audio/microwakeword/llext/CMakeLists.txt	Adds LLEXT build for the MWW module and its private TFLM library.
src/audio/microwakeword/llext-wrap.c	Adds LLEXT portability stubs and PIC-safe math overrides.
src/audio/microwakeword/Kconfig	Adds Kconfig options for MWW component and debug/model-loading modes.
src/audio/microwakeword/CMakeLists.txt	Adds static + LLEXT build logic for MWW and its private deps.
src/audio/mfcc/tune/setup_mfcc.m	Adds MFCC export profiles for mel40 (20ms) and mel40_10ms (+ compress variants).
src/audio/mfcc/mfcc_common.c	Clarifies VAD input comment.
src/audio/Kconfig	Includes microwakeword Kconfig in audio menu.
src/audio/CMakeLists.txt	Adds microwakeword subdir to audio build when enabled.
src/audio/buffers/ring_buffer.c	Adds vregion refcount release for DP→DP bind case.
src/audio/buffers/audio_buffer.c	Adds DP→DP dual-secondary-buffer sync support and updates DP→DP commentary.
scripts/xtensa-build-zephyr.py	Installs symlinks by UUID name for library binaries.
scripts/tensorflow-clone.sh	Refactors dependency clone script to fetch + checkout pinned commits under workspace parent dir.
scripts/llext_offset_calc.py	Handles ELF with no allocated sections by returning size 0.
app/llext_relocatable.conf	Enables export-by-SLID for LLEXT relocatable builds.
app/boards/intel_adsp/Kconfig.defconfig	Enables DP→DP bind by default.
app/boards/intel_adsp_cavs25.conf	Enables MWW (static), increases heap sizes, and adds related settings for cavs2.5.
app/boards/intel_adsp_ace30_ptl.conf	Enables MWW as LLEXT, adjusts library base address, heap sizing, and LLEXT heap.

Review details

Files reviewed: 86/88 changed files
Comments generated: 4
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

When CONFIG_LLEXT_TYPE_ELF_RELOCATABLE is active, bypass appending static address flags (-Ttext, --section-start, -Tdata) in the linker helper script. This keeps section base addresses at 0. Also adjust the offset calculator to avoid integer parsing errors when all section addresses are set to 0. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

…table modules Implement page-level virtual memory mapping using Zephyr's sys_bitarray utility over the library region. Compile section layout at load-time to allocate virtual addresses and rewrite section sh_addr headers in-place. This enables Zephyr LLEXT to naturally relocate references. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Without this flag, calls between GLOBAL-visibility functions defined in different translation units of the SAME llext module are emitted as PLT calls. Zephyr's llext_link_plt() only resolves PLT symbols against the base image's export table, this module's own .exported_sym table, or other already-loaded extensions -- never against symbols merely defined locally in this module's own .dynsym. Multi-TU C++ libraries (e.g. TensorFlow Lite Micro) call plenty of non-exported internal helpers across .cc files, so without this flag those calls fail to link at load time. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

LLEXT modules linking C++ code (e.g. TensorFlow Lite Micro) can end up with undefined references to global operator new/delete and __cxa_pure_virtual even when built with -fno-exceptions -- some support code (e.g. TFLM's arena allocators) still emits calls to the sized deallocation form in generated destructors, and any TU referencing an abstract class's vtable needs __cxa_pure_virtual resolvable for its pure-virtual slots. zephyr/lib/cpp/minimal/cpp_new.cpp and cpp_virtual.c already define these, but neither is referenced anywhere in the base image build, so their definitions are never pulled into the link or exported. Add thin wrappers in src/lib/cpp_new_export.cpp and export them under the mangled names so LLEXT modules can resolve against the base image. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Copilot

🟡 Changes recommended

The review found concrete correctness issues in new/changed runtime code paths (heap free mismatch in ring buffer cleanup, unsafe AMS memcpy_s handling, and incorrect syscall stub signatures/errno semantics) that should be fixed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 85/87 changed files
Comments generated: 4
Review effort level: Lite

singalsu · 2026-09-04T06:58:52Z

+	if (alloc && alloc->vreg) {
+		if (!vregion_put(alloc->vreg))
+			rfree(alloc);
+	}


This commit is do-not-merge [DNM]. The actual DP-DP bind commit is #10562 .

Export ams_send(), ams_helper_register_producer(), ams_helper_unregister_producer(), and ams_helper_prepare_payload() so an LLEXT module can act as an AMS message producer (e.g. a keyword-spotting component signaling KPB directly) without needing these calls statically linked into the base image. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Consume the lib_uuids dict already populated earlier in the script: when a library's final .bin file did not yet exist at UUID-collection time, its UUIDs were deferred into lib_uuids instead of being symlinked immediately. install_lib() now walks lib_uuids[key] and creates the deferred <uuid>.bin symlink/copy once the library is actually installed. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Copilot

🟡 Changes recommended

It introduces an unsafe free path in ring_buffer_free() that can corrupt module allocation lifetime (use-after-free/double-free risk).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/audio/buffers/ring_buffer.c:113

ring_buffer_free() must not free the module allocation context. alloc is owned by the module (allocated with sof_heap_alloc() in module_adapter_mem_alloc() and freed in module_adapter_mem_free()). Freeing it here via rfree() can cause use-after-free/double-free when the module later tears down, and rfree() is also not the matching deallocator for sof_heap_alloc().

	if (alloc && alloc->vreg) {
		if (!vregion_put(alloc->vreg))
			rfree(alloc);
	}

Files reviewed: 85/87 changed files
Comments generated: 1
Review effort level: Lite

Copilot

🟡 Changes recommended

The DP-to-DP ring buffer teardown path currently risks incorrect/freeing of shared allocation context and AMS slot message pointer rebinding relies on an invalid cross-core pointer value.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/audio/buffers/ring_buffer.c:113

ring_buffer_free() drops the vregion refcount and then frees the shared mod_alloc_ctx with rfree(). This ctx is allocated with sof_heap_alloc() (see module_adapter_mem_alloc()) and is also freed via sof_heap_free() in other vregion_put() call sites (e.g. comp_buffer_free()), so using rfree() (and freeing it here at all) risks allocator mismatch or double-free when DP-to-DP bindings are torn down.

	if (alloc && alloc->vreg) {
		if (!vregion_put(alloc->vreg))
			rfree(alloc);
	}

Files reviewed: 86/87 changed files
Comments generated: 1
Review effort level: Lite

 	msg = shared_c->slots[slot].u.msg;
+	if (msg.message && msg.message_length > 0) {
+		if (msg.message_length <= sizeof(msg_buf)) {
+			if (memcpy_s(msg_buf, sizeof(msg_buf),
+				     (__sparse_force void *)(shared_c->slots[slot].u.msg_raw + sizeof(msg)),
+				     msg.message_length) != 0) {
+				ams_release(shared_c);
+				return -EINVAL;
+			}
+			msg.message = msg_buf;
+		} else {
+			msg.message = (__sparse_force uint8_t *)(shared_c->slots[slot].u.msg_raw + sizeof(msg));
+		}
+	}


Call scheduler_dp_init() in platform_init() on cAVS platforms when CONFIG_ZEPHYR_DP_SCHEDULER is enabled so that DP tasks (such as MFCC and MWW in the Data Processing domain) can bind to the DP scheduler without failing with -ENODEV (-19). Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

…tend Add 40-bin mel filterbank configurations (mel40, mel40_compress, mel40_10ms, and mel40_10ms_compress) to setup_mfcc.m and generate the corresponding topology blobs for microWakeWord streaming frontends. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Add microWakeWord (MWW) processing module for low-power keyword spotting: - Implement module adapter in mww.c with soft mel-log AGC, VAD gating, and KPB wake-on-voice notification. - Implement TFLM bridge in mww_model.cc with MixConv operator resolver, support for int8 dequantization, and streaming resource variable resets. - Add robust circular ring buffer boundary unwrap handling for MFCC hops. - Add 2-step debounce verification before triggering KPB drain. - Add LLEXT wrapper and build system integration for both static and dynamic module targets. - Include initial placeholder model interface in mww_model_data.h/cc. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Update the embedded strawberry wake-word model data array with the retrained causal streaming MixConv model. Streaming verification report (threshold 0.65): ================================================================= microWakeWord Streaming Verification Report (Threshold: 0.65) ================================================================= Class Role Files Detected Rate Mean Peak ------------------------------------------------------------- silence Negative 2000 0 0.0% 0.000 unknown Negative 25000 21 0.1% 0.002 strawberry Positive 3000 2999 100.0% 0.996 ------------------------------------------------------------- Overall Wake Word Recall (True Positive Rate) : 99.97% (2999/3000) Overall False Alarm Rate (False Positive Rate): 0.08% (21/27000) Precision : 99.30% ================================================================= Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

…hain Add an end-to-end offline training, quantization, and verification toolchain for microWakeWord streaming models using SOF host testbench MFCC features: - sof_mfcc_extract_features.sh: Batch-extract real SOF 40-bin mel spectrogram features from WAV datasets via sof-testbench4. - sof_mww_generate_keyword_dataset_piper_tts.sh: Synthesize keyword utterances across multiple Piper ONNX neural voices. - sof_mww_prepare_silence_unknown.sh: Prepare ambient background and non-target speech datasets. - sof_mww_dataset.py: Dataset loader with temporal jitter, silence pool background mixing, and hard negative fragment synthesis. - sof_mww_train.py: Causal streaming MixConv model training with quantization-aware calibration and C-array / topology export. - sof_mww_verify.py: Streaming temporal verification with configurable threshold and consecutive-detection debounce. - sof_mww_train_pipeline.sh: One-shot automation pipeline for training, export, and streaming verification. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

…d pipelines Add Topology2 configuration and pipeline graphs for microWakeWord (MWW) Wake-on-Voice with KPB: - Define MWW module component in mww.conf with default control definitions. - Add HDA Mic capture pipeline (host-gateway-src-mfcc-mww-capture.conf) and host-gateway-micsel-mfcc-mww-capture.conf. - Add SoundWire jack and DMIC MWW branches and DMIC MFCC profiles. - Add HDA generic and SoundWire MTL/ARL topology targets with KPB and MWW. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Include audio/microwakeword/mww.toml when CONFIG_COMP_MWW is enabled across platform rimage manifest headers (tgl, tgl-h, mtl, lnl, ptl, wcl) so the MWW module UUID and entry are registered in base firmware images and loadable on target devices. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Enable CONFIG_COMP_MWW_MODEL_FROM_CONTROL in the cAVS 2.5 board configuration so that the microWakeWord model is loaded from runtime bytes control instead of relying on the built-in static C array. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

This patch adds blobs and C model files for additional keywords "banana" and "orange" for testing purposes. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Binding two DP (Data Processing) scheduled components was previously rejected with IPC4_INVALID_REQUEST because both sides required a secondary ring buffer. This patch adds support for DP-to-DP component binding under a new CONFIG_DP_TO_DP_BIND Kconfig option. In a DP-to-DP connection, a single shared ring buffer is created and attached as a secondary buffer on both the source and sink sides of the intermediate comp_buffer. The upstream DP module writes directly to the ring buffer sink API, and the downstream DP module reads directly from its source API. No copying or intermediate synchronization is required during low-latency (LL) scheduling cycles. The DP module virtual memory region backing the ring buffer is refcounted so that it remains valid across component lifetimes, and audio buffer reset and free operations ensure the shared secondary buffer is not reset or freed twice. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

When a pipeline is being stopped or unbound, the component unbind handler may clear pipeline->source_comp before the low-latency copy task finishes its final execution tick. Attempting to access p->source_comp->direction without checking for NULL triggers a synchronous PIF exception (DSP panic). Add NULL checks for p->source_comp and start before proceeding with the pipeline graph copy walk, returning 0 cleanly if the pipeline endpoints are no longer valid. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The intermediate buffer connecting KPB to micsel previously defaulted to 256 bytes (2 ms at 16 kHz stereo 32-bit). When microWakeWord runs inference on Core 1, the core is occupied for roughly 19.5 ms, starving micsel from consuming data. This caused the 2 ms buffer to fill up quickly and back-pressured KPB on Core 0, resulting in choppy, spliced audio in the Wake-on-Voice history buffer. Configure input and output buffer sizes on the micsel widget to 1280 bytes (10 ms) so that IPC4 allocates a 20 ms buffer (2560 bytes). This absorbs the inference processing latency without stalling KPB. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The soft mel-log AGC release rate during silence (VAD == 0) was previously set to 4194 (~0.5 dB/s). After loud close-mic speech, the slow recovery kept the gain heavily attenuated for several seconds, causing subsequent softer utterances to fail detection. Double the silence release step to 8389 (~1.0 dB/s) while keeping the speech leak rate unchanged. This allows faster gain recovery during pauses between phrases without affecting active utterance stability. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Lower the wake-word probability detection threshold from 0.65 to 0.60 to improve trigger sensitivity on keywords such as banana, where slight accent or acoustic variations can yield peaks in the 60-65% range. Also update the default threshold in the mtrace diagnostic plotting script to match. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The generic widget-common.conf defaults (24576 bytes heap, 8192 bytes stack) are sized for the general case, but Cadence codecs running as DP modules under CONFIG_SOF_USERSPACE_APPLICATION need more of both: - The codec's own memory tables (persist/scratch/input/output) can exceed the default heap budget on their own. On top of that, every buffer connecting to a DP module is also carved out of that same module's vregion (ipc4_comp_connect() allocates from dp->mod->priv.resources.alloc), so the requirement has to cover the codec's memory tables plus all of its connected buffers, not just the codec. - Codec init (e.g. xa_aac_dec, xa_mp3_enc) can need more stack than the default when run inside the DP userspace thread. Bump stack_bytes_requirement to 16384 for both decoder and encoder widgets. Bump heap_bytes_requirement to 196608 for the decoder: AAC and MP3 fit within 131072, but FLAC's own persist/scratch/input/output tables plus its DP-boundary ring buffers measure ~152KB, so 131072 isn't enough once FLAC or PCM are exercised. Encoder is left at 131072, unverified against a codec that needs more. Verified on hardware: AAC, MP3, FLAC and PCM decode all work. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

llext_manager_add_domain()/llext_manager_rm_domain() crashed (llext_manager_mod_find() dereferencing an invalid ctx) when called for a native (non-LLEXT) module scheduled on the DP+userspace-application path. lib_id == 0 is reserved and never populated by a loaded library, but was not guarded against. Skip the LLEXT domain add/remove entirely when LIB_MANAGER_GET_LIB_ID(module_id) == 0. llext_manager_mod_find() also needs its own guard against ctx->n_mod == 0: that's reachable for a genuine (non-NULL) library context, e.g. before its module segments are registered, and without the check the i == 0 case in the loop below reads ctx->mod[-1], out of bounds. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

cfg->ext_data points to a stack variable in module_adapter_new_ext(), valid only synchronously - but SOF_IPC4_MOD_INIT_INSTANCE runs asynchronously in the DP userspace thread, so ops->init() ended up reading a stale pointer into a different thread's stack (privilege violation, not a simple NULL deref). Add ext_data to union scheduler_dp_thread_ipc_param, flatten it by value in ipc_thread_flatten() (copying it while still on the calling thread, before the pointer goes stale), and repoint pmod->priv.cfg.ext_data at the flattened copy in ipc_thread_unflatten_run() before calling ops->init(), clearing it again afterwards to match the existing "not valid outside init()" convention. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

cadence_api_table[] is a compile-time-constant lookup table (codec id -> function pointer), populated once at link time and never written to at runtime, but declared without const - placing it in .data. Zephyr's Xtensa MMU marks .text/.rodata XTENSA_MMU_MAP_SHARED (globally accessible from every memory domain) but not .data, so the DP userspace thread's restricted domain couldn't reach it, causing a privilege-violation crash on the very first codec dispatch. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

z_impl_mod_balloc_align() called sof_heap_alloc(res->alloc->heap, ...) directly, bypassing the module's vregion entirely - unlike mod_alloc_ext(), which correctly checks vreg first via sof_ctx_alloc(). For DP/vregion-scoped modules res->alloc->heap is NULL, so this silently fell through to the global heap/rballoc(), handing back memory outside the DP thread's memory domain. This is what made cd->self (a codec's own library object, allocated via mod_balloc()) unreachable from inside the DP userspace thread. Route through sof_ctx_alloc(res->alloc, ...) like mod_alloc_ext() already does. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

DP components cannot access pipeline objects. Move the End Of Stream flag to the component type. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

struct vregion is created on the DP module's own core (via vregion_create(), invoked from module_adapter_dp_heap_new()) but is read and written from other cores too - e.g. buffer_new() on the IPC core dereferences vr->lifetime.{base,size,used,ptr} for every buffer connecting to that DP module. The struct was allocated with plain rmalloc(0, ...), i.e. ordinary cached memory. Writes made by the creating core can stay in that core's D-cache indefinitely, invisible to any other core reading the same physical address, since nothing ever flushes them out. In practice this showed up as buffer_new() on the IPC core intermittently seeing an all-zero vregion (base/size/used all 0), so any allocation failed as "lifetime alloc failed ... free 0" even though the vregion had over 100KB genuinely free - the metadata core 0 read simply hadn't propagated from core 1's cache yet. Allocate the vregion metadata with SOF_MEM_FLAG_KERNEL | SOF_MEM_FLAG_COHERENT instead, matching the existing pattern used for other cross-core shared structures in this codebase (e.g. vpage.c, regions_mm.c). This returns an uncached alias, so every core sees writes immediately without needing explicit cache maintenance. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

- app/boards/intel_adsp_ace30_ptl.conf: promote COMP_IIR/COMP_SRC/MATH_IIR into base FW (option C) so pcm101 pipeline reaches MWW init. - src/audio/microwakeword/mww.c: defensive mww_prepare under CONFIG_COMP_MWW_MODEL_FROM_CONTROL (inert, flag=n). - tools/topology/topology2/include/components/eqiir.conf: bump heap_bytes_requirement/stack_bytes_requirement defaults. - zephyr/lib/alloc.c: downgrade two runtime-stats tr_info -> tr_dbg. Temporary commit; do not upstream.

The AMS/IDC path used by MWW to notify KPB about a detected keyword is not reliable when the two components live on different cores on upstream Zephyr: the wake path goes through p4wq and can crash under DP context. Add a small dcache-managed slot (kpb_notify_slot) written by the detector via kpb_notify_request_drain() and polled by kpb_copy() while KPB is in RUN state. When a pending drain request is picked up, KPB kicks off draining with a synthetic client descriptor. Make kpb_copy() defensive on IPC4 for two states where sinks can go away asynchronously. In RUN, sel_sink may still be NULL because kpb_bind() has not connected the detector pipeline yet; buffer input into history and stop the copy chain instead of returning an error. In HOST_COPY, host_sink can disappear when the host copier tears down mid-drain (arecord stopped, xrun recovery); fall back to RUN and return 0 so the kernel can pause/reset the pipeline cleanly instead of failing the free with a stuck ACTIVE state. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Replace the AMS/IDC-based mww_notify_kpb() implementation with a call to kpb_notify_request_drain(). The AMS path went through p4wq wake-up across cores, which is unsafe from DP context on upstream Zephyr and had been observed to fault when the detector and KPB run on different cores. The new polling notify slot handles the cross-core case with a plain dcache flush + poll in kpb_copy(), so mww only has to publish the requested drain time and return. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The module_adapter DP period helper derives its scheduling period from the sink's rate and free space. For MFCC the sink is a phrase-detect / feature stream whose rate is not yet propagated at prepare time, so the derived period would be bogus (or zero) and the DP thread would be scheduled at the wrong cadence. Compute dev->period from the FFT hop size and source rate directly in mfcc_prepare(). This gives module_adapter a valid override before it inspects the sinks, and matches the natural cadence at which the MFCC component produces feature frames. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

module_adapter_calculate_dp_period() divides by sink_get_frame_bytes() * sink_get_rate() for every sink of the module. Phrase-detect / event modules such as microWakeWord expose sinks that carry no audio data: their rate and frame size are zero, which crashes the DP scheduler with a divide-by-zero. Skip any sink whose frame_bytes or rate is zero. Modules with such sinks are expected to set dev->period themselves (as MFCC now does based on its FFT hop cadence). Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

sink_get_free_frames() unconditionally divided by sink_get_frame_bytes(), assuming the format had been fully propagated by the time a module queried the sink. That does not hold for component-to-component sinks whose format is set only after the upstream component finishes its own prepare (e.g. SRC->KPB before KPB publishes its input buffer format), leading to a divide-by-zero. Return 0 when frame_bytes is zero, mirroring the guard that already exists in source_get_data_frames_available(). Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

vregion.c uses EXPORT_SYMBOL() but only picked the macro up indirectly through other headers. When those headers stop pulling llext/symbol.h in (e.g. depending on Kconfig knobs) the build fails with an implicit declaration. Include the header directly. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The mel40_10ms_compress profile is used by the microWakeWord capture pipelines, where the MFCC widget lives inside a compress/encoder stream. The kernel currently cannot map an encoder-hosted MFCC back to a topology widget for SOF_IPC4_MODULE_NOTIFICATION events, so every VAD state change floods the mailbox with unmatched notifications and eventually times out MOD_SET_DX at teardown. Set update_controls = false for this profile so the firmware does not emit those notifications at all. Regenerate the exported blob (byte 142 flips from 0x01 to 0x00). Other profiles that ship the same date comment are refreshed by the same octave run and are otherwise unchanged. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The MFCC VAD kcontrol widget was created to expose the phrase-detect state to user space, but the mel40_10ms_compress blob used by the microWakeWord capture pipelines no longer emits VAD control updates. Leaving the mixer in the topology defines an ALSA control that never changes and, worse, keeps the widget listed as a notification target that the kernel cannot resolve for encoder-hosted MFCC. Remove the "MFCC VAD" mixer from the four MWW capture topologies (HDA mic and the SDW DMIC/Jack variants, plus the shared host-gateway-src-mfcc-mww capture pipeline). Non-MWW MFCC pipelines that still ship their own VAD control are left untouched. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

Switch the PTL board configuration to a cavs25-style static build of the microWakeWord capture pipeline. LLEXT, the library manager and userspace are turned off, and the modules used by the capture pipeline (MWW, MFCC, KPB, IIR, SRC, and the standard hda-generic components) are compiled into the base firmware image. This avoids the LLEXT loading path that has been unstable when combined with cross-core DP scheduling and TFLM initialization. Bump CONFIG_IDC_TIMEOUT_US to 100 ms so a CREATE_PIPELINE forwarded to a secondary core does not time out while the DP scheduler brings up the KPB/MFCC/MWW chain and TFLM allocates its tensor arena. Enable the MWW debug/trace helpers and the model-from-control path that the capture pipeline expects, and drop the obsolete LLEXT and userspace knobs. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

The runtime microphone capture is dominated by near-stationary broadband self-noise, and the wake word rides only a few dB above that floor. The piper-based positive-sample generator did not mix in any additive noise, so the model saw no examples of the keyword at low SNR against broadband noise and generalized poorly to noisy capture conditions. Add an additive-noise stage to the piper pipeline that mirrors the recorded-audio pipeline. Both pipelines now bias the noise-file selection toward the stationary broadband entries in Google Speech Commands v2 _background_noise_ (white/pink/exercise_bike/running_tap) via NOISE_STATIONARY_BIAS, which better matches the runtime floor while still leaving weight on transient noises for false-accept robustness. Defaults for the new piper stage cover SNR 0 to 25 dB with 90 percent probability; overrides via env keep the from-dir pipeline behavior unchanged. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

================================================================= microWakeWord Streaming Verification Report (Threshold: 0.65) ================================================================= Class Role Files Detected Rate Mean Peak ------------------------------------------------------------- silence Negative 500 2 0.4% 0.020 unknown Negative 1500 18 1.2% 0.019 strawberry Positive 3000 3000 100.0% 0.996 ------------------------------------------------------------- Overall Wake Word Recall (True Positive Rate) : 100.00% (3000/3000) Overall False Alarm Rate (False Positive Rate): 1.00% (20/2000) Precision : 99.34% ================================================================= ================================================================= microWakeWord Streaming Verification Report (Threshold: 0.65) ================================================================= Class Role Files Detected Rate Mean Peak ------------------------------------------------------------- silence Negative 500 46 9.2% 0.126 unknown Negative 1500 9 0.6% 0.013 banana Positive 3000 2940 98.0% 0.985 ------------------------------------------------------------- Overall Wake Word Recall (True Positive Rate) : 98.00% (2940/3000) Overall False Alarm Rate (False Positive Rate): 2.75% (55/2000) Precision : 98.16% ================================================================= ================================================================= microWakeWord Streaming Verification Report (Threshold: 0.65) ================================================================= Class Role Files Detected Rate Mean Peak ------------------------------------------------------------- silence Negative 500 35 7.0% 0.082 unknown Negative 1500 25 1.7% 0.027 orange Positive 3000 3000 100.0% 0.996 ------------------------------------------------------------- Overall Wake Word Recall (True Positive Rate) : 100.00% (3000/3000) Overall False Alarm Rate (False Positive Rate): 3.00% (60/2000) Precision : 98.04% ================================================================= Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

singalsu force-pushed the mww_mfcc_kpb_development branch from 2151ccb to 28a11a0 Compare August 27, 2026 16:51

singalsu commented Aug 28, 2026

View reviewed changes

Comment thread app/boards/intel_adsp_ace15_mtpm.conf

singalsu force-pushed the mww_mfcc_kpb_development branch from 28a11a0 to 2674e49 Compare August 28, 2026 16:37

lyakh reviewed Aug 31, 2026

View reviewed changes

Comment thread scripts/llext_link_helper.py Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from 2674e49 to c036f90 Compare September 3, 2026 14:18

singalsu requested a lite review from Copilot September 3, 2026 14:20

Copilot started reviewing on behalf of singalsu September 3, 2026 14:21 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh

Comment thread tools/topology/topology2/include/components/kpb.conf

Comment thread src/lib/ams.c

Comment thread src/audio/microwakeword/tune/sof_mww_generate_keyword_dataset.sh Outdated

lrgirdwo added 4 commits September 3, 2026 18:11

singalsu force-pushed the mww_mfcc_kpb_development branch from c036f90 to 56a1ad9 Compare September 3, 2026 16:21

singalsu requested a lite review from Copilot September 3, 2026 16:23

Copilot started reviewing on behalf of singalsu September 3, 2026 16:26 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

lrgirdwo added 2 commits September 4, 2026 10:27

singalsu force-pushed the mww_mfcc_kpb_development branch from 56a1ad9 to f518867 Compare September 4, 2026 07:30

singalsu requested a lite review from Copilot September 4, 2026 07:32

Copilot started reviewing on behalf of singalsu September 4, 2026 07:33 View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

Comment thread src/audio/microwakeword/tune/sof_mww_prepare_silence_unknown.sh Outdated

singalsu force-pushed the mww_mfcc_kpb_development branch from f518867 to 35f972b Compare September 4, 2026 16:18

singalsu requested a lite review from Copilot September 4, 2026 16:19

Copilot started reviewing on behalf of singalsu September 4, 2026 16:21 View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

singalsu added 4 commits September 8, 2026 15:08

singalsu and others added 29 commits September 8, 2026 15:08

audio: microwakeword: Add models data for additional two keywords

4bc2cc8

This patch adds blobs and C model files for additional keywords "banana" and "orange" for testing purposes. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>

pipeline: move End Of Stream to component

02646b7

DP components cannot access pipeline objects. Move the End Of Stream flag to the component type. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

singalsu force-pushed the mww_mfcc_kpb_development branch from a6eed3b to d36213c Compare September 11, 2026 15:47

Navigation Menu

Uh oh!

audio: microwakeword: add mww component - #11135

audio: microwakeword: add mww component#11135
singalsu wants to merge 39 commits into
thesofproject:mainfrom
singalsu:mww_mfcc_kpb_development

singalsu commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

singalsu Sep 4, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

singalsu commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

singalsu Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants