You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Whisper.cpp supports webasm and even has some online demos running into browser.
I didn't research that, but it seems that Unity support native C++ plugins for WebGL builds. So in theory, you can use cmake to compile whisper.cpp into .bc and embed it into web unity build.
Also Unity doesn't support microphone input for WebGL out of the box and you need third-party plugin to record anything.
Did some tests today. Here is rough list of tasks that need to be done:
Unity doesn't support C# multithreading. Whisper blocks main unity thread and app hangs until inference complete. The good news: Unity 2022.2+ supports C++ multithreading. If I add parallel job creation in whisper.cpp code - this should work.
Bump Unity project to 2022 LTS
Inference time is very slow comparing with official whisper.cpp demo. Unity transcribe "jfk.wav" for 15 seconds, while official demo works for 3 seconds. Researching this.
No microphone support. Need to look for some MIT license compatible microphone library for Unity.
If you want to play around with it, check webgl branch.
Reacted by dnnkeeper, Michael Bodekaer Jensen and janmajayamohanty-inteligen
Web GL builds are currently failing due to Runtime/Utils.MicrophoneRecord.cs making use of the Microphone Unity class which is not available in Web GL. These usages should be wrapped around #if !UNITY_WEBGL to allow for Web GL builds to succeed.
Activity
Macoron commentedon Jun 25, 2023
Did some tests today. Here is rough list of tasks that need to be done:
If you want to play around with it, check webgl branch.
Bodekaer commentedon May 12, 2024
Hi @Macoron, any updates on webgl support. Did you get it running?
Macoron commentedon May 13, 2024
Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.
D4N005H commentedon May 14, 2024
Based on this source, Async/Await is possible for WebGL with this addon: https://github.com/Cysharp/UniTask
janmajayamohanty-inteligen commentedon Jul 26, 2024
Got it working on Unity 2022.3.39f1. Thanks
janmajayamohanty-inteligen commentedon Jul 26, 2024
The Webgl library does not implement whisper_init_from_file_with_params . It would be of immense help to have the Latest library for webgl.
jiaxinnnnnc commentedon Oct 1, 2024
can i just check if this has been resolved and it can now work with webgl? if so how can i get it to work, sorry i am new to all this
Thaina commentedon Dec 17, 2024
Is it possible to add https://github.com/bnco-dev/unity-webgl-microphone in the meantime
Nileneb commentedon Dec 24, 2024
There is a framework, which supports async operations in unity: https://github.com/Cysharp/UniTask.git
DanielaAuroch commentedon Jan 20, 2025
Web GL builds are currently failing due to
Runtime/Utils.MicrophoneRecord.csmaking use of theMicrophoneUnity class which is not available in Web GL. These usages should be wrapped around#if !UNITY_WEBGLto allow for Web GL builds to succeed.