← all repositories

whylabs/langkit

An open-source toolkit for extracting metrics and signals from LLM inputs and outputs to enable production monitoring and safety evaluation.

990 stars Jupyter Notebook LLMOps · EvalLanguage Models
langkit
Velocity · 7d
+0.9
★ / day
Trend
steady
star history

LangKit is an open-source text metrics toolkit for monitoring language models. It extracts signals from input and output text including text quality, relevance, sentiment, toxicity, and security patterns. The toolkit provides out-of-the-box metrics for detecting jailbreaks, prompt injection attacks, hallucinations, and refusals. It integrates with the whylogs data logging library for production observability.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.