Skip to content
Merged
5 changes: 5 additions & 0 deletions src/audio/module_adapter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,11 @@ sequenceDiagram
deactivate DP
```

A sinkless DP module derives its period from the first source's
`min_available` value (its IBS) and audio format. This schedules input-only
detectors according to data arrival; a zero or otherwise invalid period is
rejected by the normal minimum-period check during prepare.

## Error Handling and Memory Sandboxing

* **Sandboxing (`mod_balloc_align`, `z_impl_mod_fast_get`, `z_impl_mod_free`)**: Since third-party DSP code is treated as semi-untrusted in memory lifetimes, module allocations grab slices from a dedicated component `dp_heap_user` heap instead of the global system heap (`mod_heap_info`). The wrapper automatically prunes leaked objects (`mod_free_all(mod)`) during teardown by keeping an `objpool` of all resource containers.
Expand Down
21 changes: 18 additions & 3 deletions src/audio/module_adapter/iadk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,19 @@ classDiagram

Because the actual module resides in an external binary, it requires a "System Agent" to correctly instantiate the C++ objects during the component's `init` phase.

The adapter accepts IADK API 4.5.0 and 4.5.2 binaries. API 4.5.2
extends the detector interface with `WritePattern()`, adds
`SystemAgentInterface3::GetBssBase()`, and appends a versioned lookup
entry to the system service table. The existing table and vtable slots
keep their API 4.5.0 layout.

1. The OS host driver sends an IPC `INIT_INSTANCE` command for the module.
2. The `system_agent_start()` function intercepts this, invokes the dynamic module's `create_instance` entry point (which invokes a `ModuleFactory`).
3. The `SystemAgent` deduces the pin count (interfaces) and initial pipeline configurations using `ModuleInitialSettingsConcrete`.
4. The factory allocates the concrete algorithm and checks it back into SOF through `SystemAgent::CheckIn`.
3. If the entry point provides a null module placeholder, the `SystemAgent`
resolves it to the instance BSS allocated by the library manager after
verifying that the module object fits in that region.
4. The `SystemAgent` deduces the pin count (interfaces) and initial pipeline configurations using `ModuleInitialSettingsConcrete`.
5. The factory allocates the concrete algorithm and checks it back into SOF through `SystemAgent::CheckIn`.

```mermaid
sequenceDiagram
Expand All @@ -64,6 +73,7 @@ sequenceDiagram

IPC->>SA: Trigger Mod Creation
SA->>Fac: CI invokes create_instance
SA->>SA: Resolve instance BSS placeholder
Fac->>Fac: Deduce BaseModuleCfgExt
Fac->>Mod: operator new instantiate

Expand All @@ -79,7 +89,12 @@ A significant task of `IadkModuleAdapter_Process` is converting SOF's underlying

Instead of letting the module directly touch the SOF `comp_buffer` (which could change with SOF version updates), the adapter uses the abstraction APIs (`source_get_data` / `sink_get_buffer`) and wraps them:

1. Request raw continuous memory pointers from `source_get_data()`.
1. Limit each input to one source `min_available` portion (one IBS), then
request its continuous memory pointer from `source_get_data()`.
2. Construct an `intel_adsp::InputStreamBuffer` pointing to that continuous memory chunk.
3. Call the IADK `processing_module_.Process()`.
4. Release precisely the amount of consumed data using `source_release_data()`.

Input-only detector endpoints are processed even when no audio sink is
connected. Their output descriptors remain empty, so the module can consume
and inspect each IBS without producing downstream audio.
8 changes: 6 additions & 2 deletions src/audio/module_adapter/iadk/iadk_module_adapter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ int IadkModuleAdapter::IadkModuleAdapter_Process(struct sof_source **sources,
{
int ret = 0;

if ((num_of_sources > 0) && (num_of_sinks > 0)) {
if (num_of_sources > 0) {
intel_adsp::InputStreamBuffer input_stream_buffers[INPUT_PIN_COUNT];
intel_adsp::OutputStreamBuffer output_stream_buffers[OUTPUT_PIN_COUNT];
for (int i = 0; i < (int)num_of_sources; i++) {
Expand All @@ -55,6 +55,11 @@ int IadkModuleAdapter::IadkModuleAdapter_Process(struct sof_source **sources,

intel_adsp::InputStreamFlags flags = {};
i_size = source_get_data_available(sources[i]);
size_t min_available = source_get_min_available(sources[i]);

if (min_available && i_size > min_available)
i_size = min_available;

ret = source_get_data(sources[i], i_size, (const void **)&input,
(const void **)&input_start, &input_end);
if (ret != 0)
Expand Down Expand Up @@ -255,4 +260,3 @@ void* operator new(size_t size, intel_adsp::OutputStreamBuffer* placeholder) thr
return placeholder;
}
#endif

38 changes: 33 additions & 5 deletions src/audio/module_adapter/iadk/system_agent.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ namespace intel_adsp
{
namespace system
{

/* Structure storing handles to system service operations */
const APP_TASK_DATA AdspSystemService SystemAgent::system_service_ = {
native_system_service_log_message,
Expand All @@ -47,16 +46,21 @@ const APP_TASK_DATA AdspSystemService SystemAgent::system_service_ = {
native_system_service_create_notification,
native_system_service_send_notif_msg,
native_system_service_get_interface,
native_system_service_get_interface_versioned,
};

SystemAgent::SystemAgent(uint32_t module_id,
uint32_t instance_id,
uint32_t core_id,
uint32_t log_handle) :
uint32_t log_handle,
void *module_bss,
size_t module_bss_size) :
log_handle_(log_handle),
core_id_(core_id),
module_id_(module_id),
instance_id_(instance_id),
module_bss_(module_bss),
module_bss_size_(module_bss_size),
module_handle_(NULL),
module_size_(0)
{}
Expand All @@ -79,6 +83,19 @@ void SystemAgent::CheckIn(ProcessingModuleInterface& processing_module,
log_handle = reinterpret_cast<LogHandle*>(log_handle_);
}

void SystemAgent::CheckInDetector(DetectorModuleInterface& processing_module,
ModuleHandle &module_handle,
LogHandle *&log_handle)
{
CheckIn(static_cast<ProcessingModuleInterface&>(processing_module),
module_handle, log_handle);
}

void *SystemAgent::GetBssBase(void)
{
return module_bss_;
}

int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
ModulePlaceholder *module_placeholder,
size_t processing_module_size,
Expand All @@ -88,6 +105,16 @@ int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
void **obfuscated_modinst_p)
{
ErrorCode::Type error;

if (!module_placeholder) {
if (!module_bss_ || processing_module_size > module_bss_size_)
return -ENOMEM;

module_placeholder = reinterpret_cast<ModulePlaceholder *>(module_bss_);
}

module_size_ = processing_module_size;

IoPinsInfo pins_info;
const dsp_fw::DwordArray& cfg_ipc_msg =
*reinterpret_cast<const dsp_fw::DwordArray*>(obfuscated_mod_cfg);
Expand Down Expand Up @@ -118,9 +145,11 @@ int SystemAgent::CheckIn(ProcessingModuleFactoryInterface& module_factory,
if (error != ErrorCode::NO_ERROR)
return -EINVAL;

if (!module_handle_)
return -EINVAL;

IadkModuleAdapter& module_adapter = *reinterpret_cast<IadkModuleAdapter*>(module_handle_);
*obfuscated_modinst_p = &module_adapter;
reinterpret_cast<intel_adsp::ProcessingModuleInterface*>(module_placeholder)->Init();
return 0;
}

Expand All @@ -138,7 +167,7 @@ int system_agent_start(const struct system_agent_params *params,
{
uint32_t ret;
SystemAgent system_agent(params->module_id, params->instance_id, params->core_id,
params->log_handle);
params->log_handle, params->module_bss, params->module_bss_size);
void* system_agent_p = reinterpret_cast<void*>(&system_agent);

create_instance_f ci = (create_instance_f)(params->entry_point);
Expand All @@ -155,4 +184,3 @@ extern "C" void __cxa_pure_virtual() __attribute__((weak));
void __cxa_pure_virtual()
{
}

75 changes: 60 additions & 15 deletions src/audio/module_adapter/library/native_system_service.c
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
#include <rtos/userspace_helper.h>

#define RSIZE_MAX 0x7FFFFFFF
#define IADK_NOTIFICATION_HEADER_SIZE 48
#define IADK_NOTIFICATION_USER_VAL_2_MASK 0x3fffffffU

/*! Module log level priority to sof log level conversion array */
const int log_priority_map[L_MAX] = {
Expand Down Expand Up @@ -120,24 +122,42 @@ AdspErrorCode native_system_service_create_notification(struct notification_para
uint32_t notification_buffer_size,
struct notification_handle **handle)
{
if ((params == NULL) || (notification_buffer == NULL)
|| (notification_buffer_size <= 0) || (handle == NULL))
union ipc4_notification_header header = { 0 };
uint32_t payload_capacity;
struct ipc_msg *msg;

if (!params || !notification_buffer || !notification_buffer_size || !handle)
return ADSP_INVALID_PARAMETERS;

union ipc4_notification_header header;
*handle = NULL;
params->payload = NULL;

if (notification_buffer_size < IADK_NOTIFICATION_HEADER_SIZE)
return ADSP_OUT_OF_MEMORY;

payload_capacity = notification_buffer_size - IADK_NOTIFICATION_HEADER_SIZE;
if (params->max_payload_size > payload_capacity)
return ADSP_OUT_OF_MEMORY;

if (params->type == NOTIFICATION_TYPE_MODULE_EVENT_NOTIFICATION &&
params->max_payload_size < sizeof(struct module_event_notification))
return ADSP_INVALID_PARAMETERS;

header.r.notif_type = params->type;
header.r._reserved_0 = params->user_val_1;
header.r.rsvd0 = params->user_val_1;
header.r.type = SOF_IPC4_GLB_NOTIFICATION;
header.r.rsp = SOF_IPC4_MESSAGE_DIR_MSG_REQUEST;
header.r.msg_tgt = SOF_IPC4_MESSAGE_TARGET_FW_GEN_MSG;
struct ipc_msg *msg = lib_notif_msg_init((uint32_t)header.dat, notification_buffer_size);

if (msg) {
*handle = (struct notification_handle *)msg;
params->payload = msg->tx_data;
}
msg = lib_notif_msg_init(header.dat, params->max_payload_size);
if (!msg)
return ADSP_OUT_OF_MEMORY;

msg->extension = params->user_val_2 & IADK_NOTIFICATION_USER_VAL_2_MASK;
params->payload = msg->tx_data;
params->max_payload_size = msg->tx_size;

*handle = (struct notification_handle *)msg;
return ADSP_NO_ERROR;
}

Expand All @@ -146,21 +166,45 @@ AdspErrorCode native_system_service_send_notif_msg(enum notification_target noti
struct notification_handle *message,
uint32_t actual_payload_size)
{
if ((message == NULL) || (actual_payload_size == 0))
return ADSP_INVALID_PARAMETERS;

struct ipc_msg *msg = (struct ipc_msg *)message;

switch (notification_target) {
case NOTIFICATION_TARGET_DSP_TO_HOST:
break;
default:
return ADSP_INVALID_TARGET;
}

if (!msg || actual_payload_size > msg->tx_size)
return ADSP_INVALID_PARAMETERS;

msg->tx_size = actual_payload_size;
lib_notif_msg_send(msg);
return ADSP_NO_ERROR;
}

AdspErrorCode native_system_service_get_interface(enum interface_id id,
struct system_service_iface **iface)
{
if (id < 0)
if (!iface)
return ADSP_INVALID_PARAMETERS;
return ADSP_NO_ERROR;

(void)id;
*iface = NULL;
return ADSP_SERVICE_UNAVAILABLE;
}

AdspErrorCode native_system_service_get_interface_versioned(enum interface_id id,
uint32_t version,
struct system_service_iface **iface)
{
if (!iface)
return ADSP_INVALID_PARAMETERS;

(void)id;
(void)version;
*iface = NULL;
return ADSP_SERVICE_UNAVAILABLE;
}

const APP_TASK_DATA struct native_system_service native_system_service = {
Expand All @@ -171,6 +215,7 @@ const APP_TASK_DATA struct native_system_service native_system_service = {
.vec_memset = native_system_service_vec_memset,
.notification_create = native_system_service_create_notification,
.notification_send = native_system_service_send_notif_msg,
.get_interface = native_system_service_get_interface
.get_interface = native_system_service_get_interface,
.get_interface_versioned = native_system_service_get_interface_versioned
}
};
22 changes: 20 additions & 2 deletions src/audio/module_adapter/module_adapter.c
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,18 @@ static void module_adapter_calculate_dp_period(struct comp_dev *dev)

}

/* Sinkless modules derive their deadline from one input buffer. */
if (period == UINT32_MAX && mod->num_of_sources > 0) {
size_t frame_bytes = source_get_frame_bytes(mod->sources[0]);
unsigned int rate = source_get_rate(mod->sources[0]);

if (frame_bytes && rate)
period = 1000000ULL * source_get_min_available(mod->sources[0]) /
(frame_bytes * rate);
else
period = 0;
}

dev->period = period;
}
#endif /* CONFIG_ZEPHYR_DP_SCHEDULER */
Expand Down Expand Up @@ -537,12 +549,18 @@ int module_adapter_prepare(struct comp_dev *dev)
* Hence check for NULL.
*/
sink = comp_dev_get_first_data_consumer(dev);
if (!sink) {
if (!sink && !IS_PROCESSING_MODE_SINK_SOURCE(mod)) {
comp_err(dev, "no sink present on period size calculation");
return -EINVAL;
}

mod->period_bytes = audio_stream_period_bytes(&sink->stream, dev->frames);
/* A sinkless module still needs at least one source. */
if (!sink && !comp_dev_get_first_data_producer(dev)) {
comp_err(dev, "no source or sink buffer connected");
return -EINVAL;
}

mod->period_bytes = sink ? audio_stream_period_bytes(&sink->stream, dev->frames) : 0;
comp_dbg(dev, "got period_bytes = %u", mod->period_bytes);

/* no more to do for sink/source mode */
Expand Down
2 changes: 2 additions & 0 deletions src/include/module/iadk/adsp_error_code.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ typedef uint32_t AdspErrorCode;
#define ADSP_INVALID_TARGET 142
/* Service is not supported on target platform. */
#define ADSP_SERVICE_UNAVAILABLE 143
/* Service version is not supported on target platform. */
#define ADSP_SERVICE_VERSION_UNAVAILABLE 145

/* SystemAgentInterface */
#define ADSP_MODULE_CREATION_FAILURE 16
Expand Down
28 changes: 27 additions & 1 deletion src/include/module/module/system_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,32 @@ enum interface_id {
INTERFACE_ID_SDCA = 0x1002, /* See SdcaInterface */
INTERFACE_ID_ASYNC_MESSAGE_SERVICE = 0x1003, /* See AsyncMessageInterface */
INTERFACE_ID_AM_SERVICE = 0x1005, /* Reserved for ADSP system */
INTERFACE_ID_KPB_SERVICE = 0x1006 /* See KpbInterface */
INTERFACE_ID_KPB_SERVICE = 0x1006, /* See KpbInterface */
INTERFACE_ID_TIMESTAMPING_SERVICE = 0x1007 /* See TimestampingInterface */
};

/* Public interface versions are opaque values used by the versioned GetInterface(). */
enum interface_version {
INTERFACE_VERSION_GNA_V1 = 0x1000,
INTERFACE_VERSION_INFERENCE_SERVICE_V1 = 0x1000,
INTERFACE_VERSION_INFERENCE_SERVICE_V2 = 0x1001,
INTERFACE_VERSION_SDCA_V1 = 0x1000,
INTERFACE_VERSION_ASYNC_MESSAGE_SERVICE_V1 = 0x1000,
INTERFACE_VERSION_AM_SERVICE_V1 = 0x1000,
INTERFACE_VERSION_KPB_SERVICE_V1 = 0x1000,
INTERFACE_VERSION_KPB_SERVICE_V2 = 0x1001,
INTERFACE_VERSION_TIMESTAMPING_SERVICE_V1 = 0x1000,
};

/* Versions exposed to modules built against the current IADK headers. */
#define INTERFACE_VERSION_GNA INTERFACE_VERSION_GNA_V1
#define INTERFACE_VERSION_INFERENCE_SERVICE INTERFACE_VERSION_INFERENCE_SERVICE_V2
#define INTERFACE_VERSION_SDCA INTERFACE_VERSION_SDCA_V1
#define INTERFACE_VERSION_ASYNC_MESSAGE_SERVICE INTERFACE_VERSION_ASYNC_MESSAGE_SERVICE_V1
#define INTERFACE_VERSION_AM_SERVICE INTERFACE_VERSION_AM_SERVICE_V1
#define INTERFACE_VERSION_KPB_SERVICE INTERFACE_VERSION_KPB_SERVICE_V2
#define INTERFACE_VERSION_TIMESTAMPING_SERVICE INTERFACE_VERSION_TIMESTAMPING_SERVICE_V1

/* sub interface definition.
* This type may contain generic interface properties like id or struct size if needed.
*/
Expand Down Expand Up @@ -115,5 +138,8 @@ struct system_service {
uint32_t actual_payload_size);

AdspErrorCode (*get_interface)(enum interface_id id, struct system_service_iface **iface);

AdspErrorCode (*get_interface_versioned)(enum interface_id id, uint32_t version,
struct system_service_iface **iface);
};
#endif /* MODULE_MODULE_SYSTEM_SERVICE_H */
Loading
Loading