Fine-tuning open LLMs to write SQL without enterprise GPUs

What it does DB-GPT-Hub is an experimental pipeline for specializing open-source LLMs in Text-to-SQL. It packages data preprocessing, supervised fine-tuning, prediction, and evaluation into a reusable workflow, and it ships with processed versions of public datasets including Spider, WikiSQL, BIRD-SQL, CHASE, and CoSQL.

The interesting bit The project treats SQL generation as a low-resource training problem: the README lists minimum requirements of 13.4 GB GPU RAM to fine-tune a 13B model using 4-bit QLoRA. Their baseline table shows that several base models score zero on execution accuracy without fine-tuning, while a LoRA-tuned CodeLlama-13B reaches 0.746.

Key highlights

• Supports a broad model zoo: CodeLlama, Qwen, Baichuan2, ChatGLM3, SQLCoder, and others.
• Bundles five public datasets plus a processed set derived from the NSQL template.
• Publishes execution-accuracy baselines comparing base, LoRA, and QLoRA checkpoints across easy to extra-hard query complexity.
• Recently expanded to Text2NLU and Text2GQL fine-tuning in addition to SQL.

Caveats

• The baseline table is dated December 2023 and notes it will be moved to src/dbgpt_hub_sql, suggesting the repository layout is in flux.
• The README cites a 0.825 execution accuracy from October 2023 that does not match the later baseline figures, so it is unclear which numbers reflect the current state.

Verdict Worth exploring if you want to fine-tune open models for natural-language database queries instead of calling GPT-4. Give it a pass if you need a polished inference product; this is a training workshop, not a drop-in API.