zjunlp/AutoKG
Framework for evaluating LLMs on knowledge graph construction and reasoning tasks using multi-agent systems.

This repository accompanies a WWW 2024 paper assessing large language models’ capabilities for knowledge graph construction and reasoning. It evaluates models including ChatGPT, GPT-4, and text-davinci-003 on benchmarks like RE-TACRED, DuIE2.0, and MAVEN for relation extraction and event extraction. The work also proposes using multiple agents to autonomously construct and reason over knowledge graphs, along with analyzing virtual knowledge extraction capabilities via the VINE dataset.