knuddelsgmbh/jtokkit
A Java library implementing OpenAI's tokenization encodings for counting tokens and encoding text for GPT models.

Velocity · 7d
+0.6
★ / day
Trend
→steady
star history
JTokkit is a Java implementation of OpenAI’s tiktoken tokenizer, supporting encodings like cl100k_base and o200k_base used by GPT-3.5, GPT-4, and GPT-4o. It provides an easy-to-use API for encoding and decoding text, primarily used for counting tokens before sending requests to OpenAI APIs. The library has zero dependencies and targets Java 8+.