Teaching a neural net to fish so you don't have to

What it does

This project is a local-only automation bot for the Genshin Impact fishing minigame. It uses a YOLOX model to detect fish species, locate them on screen, and find the rod landing point, while a DQN reinforcement-learning model handles the adaptive clicking needed to keep the tension bar in the optimal zone. A few fixed OpenCV pipelines handle the parts that do not need learning.

The interesting bit

The authors avoided the slog of hand-labeling thousands of fish by using semi-supervised learning to iteratively generate and refine pseudo-labels. For the RL controller, they skipped the even bigger slog of training inside a live game by building a custom simulator to pre-train the DQN, then transferring the policy to Genshin—sim-to-real for a gacha fishing bar.

Key highlights

• Hybrid architecture: YOLOX for vision, DQN for control, and traditional CV for deterministic logic.
• Semi-supervised YOLOX training bootstrapped from COCO pre-trained weights to avoid exhaustive manual annotation.
• Sim-to-real domain transfer: the DQN is pre-trained in a custom simulator before deployment in the actual game.
• Ships with pre-trained weights, so you can skip training entirely.
• Claims support for multiple screen resolutions; originally targeted 1080×1920 fullscreen.

Caveats

• README and documentation are entirely in Chinese.
• Requires Windows admin privileges and a specific Python 3.8 (or below) environment.
• Dependency installation appears fragile, with the authors noting potential timeouts and GPU driver version mismatches.

Verdict

Worth a look if you study applied RL, computer-vision game bots, or sim-to-real transfer. If you do not play Genshin Impact or care about CV pipelines, this is strictly academic voyeurism.