A 15 KB ML model that stops CJK text from breaking mid-word

What it does

BudouX is a lightweight phrase segmenter that uses a machine learning model to find sensible line-break points in languages like Japanese, Chinese, and Thai. It takes raw text or HTML and returns phrases wrapped in non-breaking markup, using zero-width spaces to prevent browsers from splitting words across lines. The whole package—including the trained model—comes in at roughly 15 KB, making it feasible to run on the client side.

The interesting bit

The project replaces its predecessor, Budou, which required bulky third-party word segmenters or calls to the Google Cloud Natural Language API. BudouX instead uses the AdaBoost algorithm to train a tiny JSON model of feature-score pairs that fits in a web bundle. You can even roll your own model for any language by feeding it a text file with phrase separators.

Key highlights

• Ships with pretrained models for Japanese, Simplified Chinese, Traditional Chinese, and Thai.
• Runs standalone: no external NLP service or heavy runtime dependency required.
• Supports HTML input, outputting markup with word-break: keep-all and zero-width space separators.
• Available as libraries for Python, JavaScript, and Java.
• Training is an anytime algorithm, so you can interrupt it and still produce a valid model.

Caveats

• It is not an HTML sanitizer—untrusted input will pass straight through to output, so pair it with a proper sanitizer.
• Korean is explicitly not supported; the README suggests using CSS word-break: keep-all instead.
• This is not an officially supported Google product.

Verdict

Worth a look if you build multilingual web frontends and want readable CJK typography without the latency or cost of a cloud segmentation service. Skip it if you need server-grade NLP accuracy or if your problem is solved by a single CSS property.