Skip to content

WebGL support #20

Open
Open

Description

@Macoron

Whisper.cpp supports webasm and even has some online demos running into browser.

I didn't research that, but it seems that Unity support native C++ plugins for WebGL builds. So in theory, you can use cmake to compile whisper.cpp into .bc and embed it into web unity build.

Also Unity doesn't support microphone input for WebGL out of the box and you need third-party plugin to record anything.

Activity

  1. Macoron commented on Jun 25, 2023

    @Macoron
    OwnerAuthor

    Did some tests today. Here is rough list of tasks that need to be done:

    • Unity doesn't support C# multithreading. Whisper blocks main unity thread and app hangs until inference complete. The good news: Unity 2022.2+ supports C++ multithreading. If I add parallel job creation in whisper.cpp code - this should work.
    • Bump Unity project to 2022 LTS
    • Inference time is very slow comparing with official whisper.cpp demo. Unity transcribe "jfk.wav" for 15 seconds, while official demo works for 3 seconds. Researching this.
    • No microphone support. Need to look for some MIT license compatible microphone library for Unity.

    If you want to play around with it, check webgl branch.

  2. Bodekaer commented on May 12, 2024

    @Bodekaer

    Hi @Macoron, any updates on webgl support. Did you get it running?

  3. Macoron commented on May 13, 2024

    @Macoron
    OwnerAuthor

    Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

  4. D4N005H commented on May 14, 2024

    @D4N005H

    Not much, biggest blocker is that Unity doesn't support C# multithreading. But you can checkout webgl branch. It's a bit outdated, but still working.

    Based on this source, Async/Await is possible for WebGL with this addon: https://github.com/Cysharp/UniTask

  5. janmajayamohanty-inteligen commented on Jul 26, 2024

    @janmajayamohanty-inteligen

    Got it working on Unity 2022.3.39f1. Thanks

  6. janmajayamohanty-inteligen commented on Jul 26, 2024

    @janmajayamohanty-inteligen

    The Webgl library does not implement whisper_init_from_file_with_params . It would be of immense help to have the Latest library for webgl.

  7. jiaxinnnnnc commented on Oct 1, 2024

    @jiaxinnnnnc

    can i just check if this has been resolved and it can now work with webgl? if so how can i get it to work, sorry i am new to all this

  8. Thaina commented on Dec 17, 2024

    @Thaina

    No microphone support. Need to look for some MIT license compatible microphone library for Unity.

    Is it possible to add https://github.com/bnco-dev/unity-webgl-microphone in the meantime

  9. Nileneb commented on Dec 24, 2024

    @Nileneb

    There is a framework, which supports async operations in unity: https://github.com/Cysharp/UniTask.git

  10. DanielaAuroch commented on Jan 20, 2025

    @DanielaAuroch

    Web GL builds are currently failing due to Runtime/Utils.MicrophoneRecord.cs making use of the Microphone Unity class which is not available in Web GL. These usages should be wrapped around #if !UNITY_WEBGL to allow for Web GL builds to succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions