NVIDIA's unofficial LLM gym: from synthetic data to Slurm

What it does

Nemo-Skills is a collection of Python pipelines that handle synthetic data generation, model training, and evaluation for large language models. It wraps multiple inference backends—TensorRT-LLM, vLLM, sglang, and Megatron—so you can host and query models locally or distribute work across a Slurm cluster. The project also bundles evaluation harnesses for domains like math, code, scientific reasoning, tool use, and vision-language tasks.

The interesting bit

The README claims you can start on a local workstation and move to large-scale Slurm jobs with a one-line change, which suggests the abstractions are built around scaling out rather than merely wrapping individual tools. It is strictly a research project, not an official NVIDIA product, which explains the direct ties to NVIDIA’s own models and datasets.

Key highlights

• Supports inference through TensorRT-LLM, vLLM, sglang, and Megatron, with the ability to switch between API providers, local servers, or Slurm jobs.
• Bundles evaluation pipelines for math (natural and formal), code, scientific knowledge, instruction following, long-context, tool-calling, multilingual, speech, and vision-language benchmarks.
• Integrates training through NeMo-RL or verl.
• Ships with reproducible recipes for NVIDIA-released datasets and models such as Nemotron-Math-v2 and OpenMathReasoning.
• Scales synthetic data generation jobs from one local GPU up to tens of thousands of cluster GPUs.

Caveats

• Explicitly marked as research-only and not an official NVIDIA product.
• Training integrations and released recipes center heavily on NVIDIA-specific stacks like NeMo-RL, TensorRT-LLM, and Megatron.
• The README is broad but light on architectural detail; it is unclear how much custom logic lives here versus how much is wiring over existing tools.

Verdict

Worth a look if you are doing LLM research inside a Slurm-managed, NVIDIA-heavy environment and want pre-built evaluation harnesses. Skip it if you need a polished, vendor-neutral product or are not prepared to navigate NVIDIA’s model-training stack.