Fine-tuning VisualGLM to read chest X-rays in Chinese

What it does

XrayGLM is essentially a fine-tuned VisualGLM-6B that ingests chest X-rays and writes diagnostic impressions in Chinese. It handles both single-shot report generation and multi-turn conversation—users can ask what the image shows, whether specific conditions like emphysema appear, or even request diet advice. The authors curated a Chinese dataset by translating public English radiology reports from OpenI and MIMIC-CXR with ChatGPT.

The interesting bit

The real labor here is dataset construction, not architecture. The team took existing public X-ray corpora, structured the messy unstructured reports, and used ChatGPT to produce Chinese diagnostic text, creating a bilingual medical vision-language dataset where none existed. The released LoRA checkpoints are explicitly labeled low-quality, and side-by-side examples show the model occasionally flatly contradicting the reference radiologist—declaring a mildly enlarged heart “normal,” for instance.

Key highlights

• Fine-tuned VisualGLM-6B with LoRA; training used 4× A100 80GB GPUs
• Two low-quality checkpoints (checkpoints-XrayGLM-300 and 3000) on Hugging Face for academic research; a higher-quality “plus” variant is mentioned but lacks a download link
• Supports both CLI and WebUI inference
• Dataset derived from OpenI (~6,423 Chinese-translated samples) and MIMIC-CXR
• CC BY-NC-SA 4.0 license—strictly non-commercial and academic

Caveats

• Showcased outputs are explicitly from low-quality weights, and the README warns that performance is limited
• Example comparisons reveal factual mismatches against the provided doctor reports
• High-quality weights and the full MIMIC-CXR-zh dataset are not yet released

Verdict

A useful reference for researchers building Chinese medical VLMs or curating radiology datasets. Not remotely clinical-ready, and the authors themselves note that a 6B model will inevitably lose to larger-parameter competitors.