I work on OpenSubs, a free, open source (AGPL-3.0) subtitle tool that runs entirely in the browser tab.
You drop in a video file, and Whisper transcribes it on your own machine, using transformers.js with WebGPU where available and WebAssembly otherwise. The model (40–250 MB) downloads once and is cached. There is no upload endpoint in the product, so the video has nowhere to go.
After that you can:
- fix lines by typing over them (click a timestamp to jump to that moment)
- translate into 20 languages with Chrome’s built-in on-device translator
- pick one of 12 caption styles, including word-by-word highlighting
- export SRT / VTT / ASS, or burn the subtitles into an MP4 (libass compiled to WebAssembly, encoded with WebCodecs)
A few things I learned building it:
- Whisper hallucinates on silence and music (“Thanks for watching!”, or the Japanese equivalent). A Silero VAD pass runs before Whisper, and a cleanup step drops the known stock phrases.
- Singing doesn’t count as speech for the VAD, so a music video gets a “no speech found” warning. You can still force it.
Honest limits: it only takes video files, not audio-only files. Cue timings can’t be edited yet. Builds are release candidates. Everything that runs locally is free with no account; the only paid part is optional cloud translation on our backend (US$5 for 1000 credits), and you can bring your own Claude / OpenAI / DeepL key instead.
Site: https://opensubs.app/ Code: https://github.com/open-subs/opensubs
Feedback welcome, especially on languages where the transcription goes wrong.
Whisper hallucinates on silence and music (“Thanks for watching!”, or the Japanese equivalent).
I’ve had Whisper ask me to donate to anime fansubbing groups. I wonder what it was trained on?
It sounds like a pretty cool project, thanks for sharing! As a browser-based project designed to run offline, you might want to consider shipping it as an electron app.
Why not just use the whisper plugin for Bazarr?
Open source, but made with a copyright infringing LLM.
Whisper is not an llm, just because it is trained on big datasets does not make it an LLM. Copyright infringement still happened during the training.
no, but claude is, and it’s the main contributor to the repo.
Whisper is not an llm, just because it is trained on big datasets does not make it an LLM.
The training dataset consists of 680,000 hours of labeled audio-transcript pairs sourced from the internet using semi-supervised learning.
Brother, I don’t think you know what a large language model is if you think it being trained on a human lifespan’s worth of audio transcripts isn’t enough.
Its job is NLP. It is an LLM, if not as general-use as something like ChatGPT.
LLM is a subset of NLP. All LLMs are NLP but not all NLP are LLMs. To be more precise, LLM is a method of achieving NLP while NLP is the goal. Whisper is just a specialised speech-to-text engine. It doesn’t have the text reasoning engine that is the core of LLMs. Yes, it has a dataset to inform it’s encoder and decoder, but that doesn’t make it an LLM. If you strip out the encoder, then you’re getting closer.
The browser runs a lot of stuff that doesn’t get passed on to the servers you connect to. In this case it can run a local whisper model locally and pass your video stream through it, rather than pass it through some server online.
@virku Then what’s the point? You can just download the model and run it locally without using this service…
It’s not a “service” but a program that runs locally.
Downloading the model and running it locally is essentially what it does. If you do it by yourself then you’d have to write your own scripts. At which point, you might as well share those scripts with the community putting them in Github, and maybe including them in a web extension to integrate with the browser and/or in a static website that locally runs it via WASM… precisely what this project is doing…
Ease of use…
If i understood this right…
Instead of uploading a file you just download the model. The sound then gets transcribed as usual without the service accessing the file.
@beyoublahaj So audio gets uploaded?
oh nah I reread the post and edited my comment. I misunderstood too ^^`
@beyoublahaj So what’s the point of this service if you can download the model yourself and use it without this website?..
You don’t have to use the website… the project is not just that website, it’s also packaged as a browser extension, a desktop app and a CLI program.
That website is just a convenient way to see it working (while still processing it locally) without installing anything.
If you download the raw model and write up scripts to create subtitles out of video by yourself then you are essentially re-building the same app… the result would be similar, just duplicating effort.
I assume you can use it on any platform that runs a browser, without installing software on the system. I use (other type of) software that runs in the browser, but locally. This type of browser software is handy.
No idea, I suppose that it’s more convenient and easier.
