Pre-submission paper review with a guilty-until-proven-safe filter

What it does

PaperJury is a Claude Code skill that acts as a pre-submission mock reviewer for LaTeX and Markdown research drafts. It runs a closed loop of review, verdict, revision, and verification, sorting AI-generated feedback into safe fixes, author-required issues, or invalid complaints before any change reaches the manuscript. If a local LaTeX toolchain exists, it compiles the draft to flag formatting errors and undefined references; if not, it says so rather than pretending to have checked.

The interesting bit

Most AI writing tools rush to edit; PaperJury inserts a deliberation stage where the agent must argue with itself. It separates “this caption is unclear” from “this claim needs a new ablation study,” and it will not auto-apply the latter even in unattended mode. The repository includes a dogfood sample with before-and-after PDFs and human-checked run reports, which is more evidence than most tools in this space bother to provide.

Key highlights

• Three modes: direct-edit for surgical text tweaks, review for adversarial mock-review, and auto for multi-round unattended polishing with author override on risky items.
• Feedback triage: every reviewer-style comment is tagged as safe fix, author-required, or invalid, so the tool does not hallucinate experiments into existence.
• LaTeX-aware: it can compile and check for overfull boxes, missing references, and desk-reject risks if the toolchain is present; otherwise it admits the gap.
• Ships with a real dogfood sample—original draft, revised draft, and a human-audited execution report—so you can inspect actual output before trusting it with your own paper.
• Also available as a Codex plugin, and the authors published an arXiv paper on the engine design.

Caveats

• Requires Node.js for deterministic checks, and a LaTeX toolchain for compilation-level validation; without them, some checks simply do not run.
• Auto mode must be explicitly enabled with /goal or mode: auto; a normal prompt runs a single round and stops, which is easy to miss if you expect continuous iteration.
• The README notes it handles .docx by converting to Markdown, but the fidelity of that conversion is unclear from the description.

Verdict

Academics writing in LaTeX who want a structured sanity check before submission will find this more disciplined than a raw chatbot. If you write in Word or expect the AI to invent missing experiments for you, this is explicitly not your tool.