A lightweight tokenizer compiled to WebAssembly. Its WebAssembly binary is under 260KiB gzipped, and the tokenizer runs entirely client-side — no server, no multi-megabyte dictionary.
Client-side Japanese tokenization, without the usual server, dictionary, and privacy trade-offs.
The WebAssembly binary is under 260KiB gzipped and includes its bundled dictionaries — no multi-megabyte dictionary to fetch or host.
Runs entirely in the browser or your runtime. No tokenization server, no API round-trip, no CORS setup.
Analysis happens on-device, so user input never leaves the browser and sensitive text stays out of your infrastructure.
Combines compact dictionaries with character and grammar patterns to handle words without a dictionary entry.
A synchronous API for local text processing, without a network round trip.
A C++ core with first-class TypeScript types, plus Python, Go, and native C/C++ bindings for backend, batch, and embedded use.
Choose an everyday, colloquial, dialectal, or classical example, or paste your own text. Processing stays in your browser.
TinySegmenter's light footprint, closer to MeCab's detail — Suzume sits between them.
| Feature | Intl.Segmenter Built-in | TinySegmenter Light | Suzume | kuromoji | MeCab Accurate |
|---|---|---|---|---|---|
| Browser | ✓ | ✓ | ✓ | △ | ✗ |
| External Dictionary Setup | Not required | Not required | Bundled | Required | Required |
| Bundle Size | 0KB | ~10KB | 260KiB WASM | ~20MB | N/A |
| Server-free | ✓ | ✓ | ✓ | △ | ✗ |
| POS Tagging | ✗ | ✗ | ✓ | ✓ | ✓ |
| Lemmatization | ✗ | ✗ | ✓ | ✓ | ✓ |
| Dictionary-defined compound splitting | ✗ | ✗ | △ | ✓ | ✓ |
| Custom Dictionary | ✗ | ✗ | ✓ | ✓ | ✓ |
| Unknown Words | △ | △ | ✓ | △ | △ |
npm install @libraz/suzumeyarn add @libraz/suzumepnpm add @libraz/suzumebun add @libraz/suzumepip install suzumegit clone https://github.com/libraz/go-suzume.git
cd go-suzume && make lib
cd /path/to/your/module
go mod edit -replace github.com/libraz/go-suzume=/path/to/go-suzume
go get github.com/libraz/go-suzumegit clone https://github.com/libraz/suzume.git
cd suzume && make installFor Python services and data pipelines, see the Python bindings guide. Suzume is also available through the Go binding and as a C / C++ library.
The library bindings expose the same analysis model with names adapted to each language:
import { Suzume } from '@libraz/suzume'
const suzume = await Suzume.create()
try {
for (const m of suzume.analyze('東京都に住んでいます')) {
console.log(m.surface, m.pos, m.baseForm)
}
} finally {
suzume.destroy()
}from suzume import Suzume
with Suzume() as sz:
for m in sz.analyze("東京都に住んでいます"):
print(m.surface, m.pos, m.base_form)package main
import (
"fmt"
"log"
"github.com/libraz/go-suzume"
)
func main() {
analyzer, err := suzume.New()
if err != nil {
log.Fatal(err)
}
defer analyzer.Close()
for _, m := range analyzer.Analyze("東京都に住んでいます") {
fmt.Println(m.Surface, m.POS, m.BaseForm)
}
}#include "suzume/suzume.hpp"
#include <cstdio>
int main() {
suzume::Tokenizer tokenizer;
for (const suzume::Morpheme& m : tokenizer.analyze("東京都に住んでいます"))
std::printf("%s\t%s\t%s\n", m.surface.c_str(), m.pos.c_str(), m.base_form.c_str());
}Each token carries a surface form, POS, base form, offsets, and more. See the JavaScript/WASM, Python, Go, or C / C++ reference for binding-specific names.