Java 21 proxy that corrals scattered LLM endpoints

What it does

DIAL Core is an HTTP proxy built on Java 21 and Eclipse Vert.x that sits between client applications and a mix of language models, embedding services, and assistants. It exposes those backends through a single API surface while handling authentication via configurable identity providers and JWT validation. The project is designed for Kubernetes deployment via Helm charts.

The interesting bit

It leans into modern JVM features, defaulting to virtual threads for blocking tasks via asyncTaskExecutor.useVirtualThreads and supporting hot-reloaded dynamic configuration so you can tweak endpoints without restarting.

Key highlights

• Java 21 and Eclipse Vert.x under the hood.
• Dynamic configuration reloads without a restart.
• JWT-based auth with pluggable identity providers and role claims.
• Kubernetes-ready via Helm charts.
• Virtual threads enabled by default for blocking workloads.

Caveats

• Building the project requires a GitHub personal access token with read:packages scope because of a dependency on JClouds packages hosted on GitHub Packages.
• Authentication is mandatory; the README states at least one identity provider must be configured, so there is no anonymous out-of-the-box mode.

Verdict

Worth a look if you are running multiple LLM and embedding backends and need a unified, auth-gated API in a Java-centric stack. Skip it if you need a lightweight, anonymous proxy or if a mandatory GitHub PAT for building sounds like too much friction.