← all repositories

knuddelsgmbh/jtokkit

A Java library implementing OpenAI's tokenization encodings for counting tokens and encoding text for GPT models.

740 stars Java Language ModelsData Tooling
jtokkit
Velocity · 7d
+0.6
★ / day
Trend
steady
star history

JTokkit is a Java implementation of OpenAI’s tiktoken tokenizer, supporting encodings like cl100k_base and o200k_base used by GPT-3.5, GPT-4, and GPT-4o. It provides an easy-to-use API for encoding and decoding text, primarily used for counting tokens before sending requests to OpenAI APIs. The library has zero dependencies and targets Java 8+.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.