A 25 MB reranker that beats Qwen at picking Chinese characters

What it does

Chinese pinyin input methods turn your keystrokes into a pool of candidate readings, and something has to choose among them. This project builds that something: a character-level Transformer that reranks candidates produced by a pinyin decoder, trained on redistributable corpora (C4, LCCC) and shipped as 4.5 MB and 25.5 MB weights on Hugging Face. There’s also an older n-gram route (KenLM + Viterbi), but the README is blunt that it never left the lab.

The interesting bit

The eval set is treated as the real product. It turns “is this change good?” from opinion into numbers, and bakes in two measurement traps as test cases. The team also benchmarked four Qwen models on the same candidate pool — none beat their 25.5 MB model, and the ranking differences were all within statistical noise (McNemar found nothing significant). They chose their own weights for size and latency, which are deterministic, not for leaderboard rank.

Key highlights

• Two Apache-2.0 reranker weights: 4.5 MB for keystroke-path use, 25.5 MB when input pauses, loaded by MSIME’s host-api
• Rust reference implementation with no unsafe and only serde as a dependency; format spec in docs/format.md is enough to write a loader in any language
• 25,119 word-level + 60 sentence-level eval cases in eval/
• Unusually careful licensing: repo is GPL-3.0, but reference/ and the weights are Apache-2.0 so MIT/BSD input methods can actually use them; corpus attribution lives in NOTICE
• Corpus scripts only fetch cleanly redistributable sources; the original wiki/news corpora’s license problems are documented rather than swept aside

Caveats

• The reranker is not universally helpful — dictionary exact hits already carry corpus word frequency, and reranking those loses points; the model’s value is only on decoder-assembled candidates
• Direct syllable decoding (reference/examples/decode.rs) still performs worse than the engine it was meant to improve
• The eval set itself contained 12 “他/她” cases that are undecidable from pinyin alone — a known flaw they acknowledge

Verdict

Worth a look for anyone building Chinese IME infrastructure, especially the eval methodology and the honest measurements doc. If you just want a drop-in pinyin engine, look at their MSIME project instead — this is the research and assets behind it.