LLM prompt assembly for token-counting obsessives

What it does Prompt Poet is a Python library that builds LLM prompts from Jinja2-templated YAML rather than raw string concatenation. You define prompt parts—system instructions, user queries, chat history—with optional roles and truncation priorities, and the library renders them into standard API message dictionaries. It wraps the messy business of token counting, context window trimming, and structured formatting into a declarative template pipeline.

The interesting bit The library treats prompts as structured data rather than text blobs. Because the YAML schema enforces depth-one parts with optional nested sections, Prompt Poet can tokenize individual components and perform priority-based truncation—including a “cache-aware” algorithm that truncates in fixed steps to maximize GPU prefix cache hits. Think of it as Ansible playbooks for chat history.

Key highlights

• Marries Jinja2 logic (loops, conditionals, arbitrary Python function calls) with YAML structure to generate dynamic prompts.
• Built-in TikToken tokenization with pluggable encoders, plus granular token statistics via nested YAML sections.
• Priority-based truncation lets you shed old chat messages before system instructions when context limits bite.
• Cache-aware truncation strategy designed to keep LLM prefix caches hot by moving truncation boundaries only every k turns.
• Decomposes templates into reusable Jinja2 includes for A/B testing and DRY maintenance.

Caveats

• The README’s explanation of cache-aware truncation is literally truncated mid-sentence, so the exact mechanics remain incomplete in the source.
• Nested section token counts may not sum exactly to the parent part’s total due to tokenization boundary effects.
• A prompt part can contain either content or sections, never both, which limits layout flexibility.

Verdict Worth a look if you manage complex, multi-turn prompts and need precise control over token budgets and inference caching. Skip it if you just need a lightweight f-string wrapper or a no-code prompt editor.