preface
The official audio plug-in of obs on windows platform has only one win wasapi, which is responsible for collecting the sound of speakers and microphones.
Win audio capture application audio output capture
Here we recommend an unofficial audio capture plug-in win audio capture, which can collect the audio of the specified process and input it to obs.
obs Forum: https://obsproject.com/forum/threads/win-capture-audio.147240/
Project address: https://github.com/bozbez/win-capture-audio
The use of this plug-in has minimum requirements for win10 version. Windows 10 2004 (released 2020-05-27) or later
Video introduction: https://www.bilibili.com/video/av676469967
Create a windows audio capture plug-in
obs audio capture plug-in is created when the program starts, including speaker sound capture wasapi_output_capture and microphone sound capture wasapi_input_capture . Both are managed through the class WASAPISource object.
The following is the call stack for creating an audio plug-in.
> win-wasapi.dll!CreateWASAPISource(obs_data * settings, obs_source * source, bool input) Line 1044 C++ win-wasapi.dll!CreateWASAPIOutput(obs_data * settings, obs_source * source) Line 1062 C++ obs.dll!obs_source_create_internal(const char * id, const char * name, obs_data * settings, obs_data * hotkey_data, bool private, unsigned int last_obs_ver) Line 387 C obs.dll!obs_source_create_set_last_ver(const char * id, const char * name, obs_data * settings, obs_data * hotkey_data, unsigned int last_obs_ver) Line 432 C obs.dll!obs_load_source_type(obs_data * source_data) Line 1781 C obs.dll!obs_load_source(obs_data * source_data) Line 1890 C obs64.exe!LoadAudioDevice(const char * name, int channel, obs_data * parent) Line 747 C++ obs64.exe!OBSBasic::LoadData(obs_data * data, const char * file) Line 1026 C++ obs64.exe!OBSBasic::Load(const char * file) Line 970 C++ obs64.exe!OBSBasic::OBSInit() Line 1893 C++ obs64.exe!OBSApp::OBSInit() Line 1474 C++ obs64.exe!run_program(std::basic_fstream<char,std::char_traits<char>> & logFile, int argc, char * * argv) Line 2138 C++ obs64.exe!main(int argc, char * * argv) Line 2839 C++ obs64.exe!WinMain(HINSTANCE__ * __formal, HINSTANCE__ * __formal, char * __formal, int __formal) Line 97 C++
Initialization of audio capture plug-in
Audio capture completes all initialization in the constructor of WASAPISource object. The callback notification of default audio device switching is also registered. win8 and above systems use Microsoft's API related to real-time scheduling work queue. Microsoft claims that if they use their API, it can better arrange audio. This is what I saw in the submission log.
For unsupported systems, a WASAPISource::CaptureThread audio collection queue is created to capture the sound of windows.
Submitted by: 24d82062ecca2f702dbf1cd7a85e84036b24ccf [24d8206]
Author: jpark37 jpark37@users.noreply.github.com Date: September 26, 2021 4:21:22
Next, post the constructor to manage the audio capture object, and analyze the creation process by annotation.
WASAPISource::WASAPISource(obs_data_t *settings, obs_source_t *source_, bool input)
: source(source_),
isInputDevice(input), // true: to create microphone audio capture false: to create speaker audio capture
startCapture(this),
sampleReady(this),
restart(this)
{
// Get the audio device id whether to use the device time whether to use the default audio device
UpdateSettings(settings);
// Creation of various event s
idleSignal = CreateEvent(nullptr, true, false, nullptr);
if (!idleSignal.Valid())
throw "Could not create idle signal";
stopSignal = CreateEvent(nullptr, true, false, nullptr);
if (!stopSignal.Valid())
throw "Could not create stop signal";
receiveSignal = CreateEvent(nullptr, false, false, nullptr);
if (!receiveSignal.Valid())
throw "Could not create receive signal";
restartSignal = CreateEvent(nullptr, true, false, nullptr);
if (!restartSignal.Valid())
throw "Could not create restart signal";
exitSignal = CreateEvent(nullptr, true, false, nullptr);
if (!exitSignal.Valid())
throw "Could not create exit signal";
initSignal = CreateEvent(nullptr, false, false, nullptr);
if (!initSignal.Valid())
throw "Could not create init signal";
reconnectSignal = CreateEvent(nullptr, false, false, nullptr);
if (!reconnectSignal.Valid())
throw "Could not create reconnect signal";
// Create a thread to reinitialize audio collection
reconnectThread = CreateThread(
nullptr, 0, WASAPISource::ReconnectThread, this, 0, nullptr);
if (!reconnectThread.Valid())
throw "Failed to create reconnect thread";
// Create the default audio device switching notification object IMMNotificationClient
notify = new WASAPINotify(this);
if (!notify)
throw "Could not create WASAPINotify";
HRESULT hr = CoCreateInstance(__uuidof(MMDeviceEnumerator), nullptr,
CLSCTX_ALL,
IID_PPV_ARGS(enumerator.Assign()));
if (FAILED(hr))
throw HRError("Failed to create enumerator", hr);
// Registration notice
hr = enumerator->RegisterEndpointNotificationCallback(notify);
if (FAILED(hr))
throw HRError("Failed to register endpoint callback", hr);
/* OBS will already load DLL on startup if it exists */
// Check whether the system supports the new api of real-time work queue (only win8 and above systems support it)
// true: supports creating work queues to schedule audio capture
// false: if it is not supported, create a CaptureThread thread to be responsible for audio capture
const HMODULE rtwq_module = GetModuleHandle(L"RTWorkQ.dll");
rtwq_supported = rtwq_module != NULL;
if (rtwq_supported) {
rtwq_unlock_work_queue =
(PFN_RtwqUnlockWorkQueue)GetProcAddress(
rtwq_module, "RtwqUnlockWorkQueue");
rtwq_lock_shared_work_queue =
(PFN_RtwqLockSharedWorkQueue)GetProcAddress(
rtwq_module, "RtwqLockSharedWorkQueue");
rtwq_create_async_result =
(PFN_RtwqCreateAsyncResult)GetProcAddress(
rtwq_module, "RtwqCreateAsyncResult");
rtwq_put_work_item = (PFN_RtwqPutWorkItem)GetProcAddress(
rtwq_module, "RtwqPutWorkItem");
rtwq_put_waiting_work_item =
(PFN_RtwqPutWaitingWorkItem)GetProcAddress(
rtwq_module, "RtwqPutWaitingWorkItem");
hr = rtwq_create_async_result(nullptr, &startCapture, nullptr,
&startCaptureAsyncResult);
if (FAILED(hr)) {
enumerator->UnregisterEndpointNotificationCallback(
notify);
throw HRError(
"Could not create startCaptureAsyncResult", hr);
}
hr = rtwq_create_async_result(nullptr, &sampleReady, nullptr,
&sampleReadyAsyncResult);
if (FAILED(hr)) {
enumerator->UnregisterEndpointNotificationCallback(
notify);
throw HRError("Could not create sampleReadyAsyncResult",
hr);
}
hr = rtwq_create_async_result(nullptr, &restart, nullptr,
&restartAsyncResult);
if (FAILED(hr)) {
enumerator->UnregisterEndpointNotificationCallback(
notify);
throw HRError("Could not create restartAsyncResult",
hr);
}
DWORD taskId = 0;
DWORD id = 0;
hr = rtwq_lock_shared_work_queue(L"Capture", 0, &taskId, &id);
if (FAILED(hr)) {
enumerator->UnregisterEndpointNotificationCallback(
notify);
throw HRError("RtwqLockSharedWorkQueue failed", hr);
}
startCapture.SetQueueId(id);
sampleReady.SetQueueId(id);
restart.SetQueueId(id);
} else {
// =======Create a CaptureThread thread to capture audio. win7 operating system takes this branch============
captureThread = CreateThread(nullptr, 0,
WASAPISource::CaptureThread, this,
0, nullptr);
if (!captureThread.Valid()) {
enumerator->UnregisterEndpointNotificationCallback(
notify);
throw "Failed to create capture thread";
}
}
// Start audio capture
Start();
}
Captured audio output
The following stack is the output stack using Microsoft's real-time scheduling work queue related api
obs_source_output_audio(source, &data); // obs audio data is asynchronous output win-wasapi.dll!WASAPISource::ProcessCaptureData() Line 772 C++ win-wasapi.dll!WASAPISource::OnSampleReady() Line 965 C++ win-wasapi.dll!WASAPISource::CallbackSampleReady::Invoke(IRtwqAsyncResult * __formal) Line 141 C++
The following stack uses CaptureThread to output the collected audio data
obs_source_output_audio(source, &data); // obs audio data is asynchronous output win-wasapi.dll!WASAPISource::ProcessCaptureData() Line 775 C++ win-wasapi.dll!WASAPISource::CaptureThread(void * param) Line 863 C++
Through obs_source_output_audio is output to the obs audio queue. For specific audio processing threads, refer to: audio_thread audio encoding thread
summary
Audio acquisition under windows is an audio acquisition work based on windows audio session related APIs. I won't teach you how to use and introduce the specific api. Please refer to Microsoft's Official documents Just.
The above are all personal understandings of OBS studio open source projects. It is inevitable that there are mistakes. If you have any, you are welcome to point out.
If it helps, I'm glad.
Technical reference
- Video technology reference: https://ke.qq.com/course/3202131?flowToken=1041285
- windows audio session : https://docs.microsoft.com/zh-cn/samples/microsoft/windows-universal-samples/windowsaudiosession/
- rtworkq: https://docs.microsoft.com/en-us/windows/win32/api/rtworkq/