michiyasunaga/LinkBERT
A BERT-style pretrained language model enhanced with document hyperlinks and citation links during pretraining for improved knowledge-intensive understanding.
Not currently ranked — collecting fresh signals.
star history
LinkBERT extends BERT pretraining by feeding linked documents (hyperlinks, citations) into the same context alongside single documents, enabling the model to capture cross-document knowledge. It serves as a drop-in BERT replacement and demonstrates improved performance on general language understanding, knowledge-intensive question answering, and cross-document tasks. Models are released in base and large sizes for both general and biomedical domains.
Frequently asked
- What is michiyasunaga/LinkBERT?
- A BERT-style pretrained language model enhanced with document hyperlinks and citation links during pretraining for improved knowledge-intensive understanding.
- Is LinkBERT open source?
- Yes — michiyasunaga/LinkBERT is open source, released under the Apache-2.0 license.
- What language is LinkBERT written in?
- michiyasunaga/LinkBERT is primarily written in Python.
- How popular is LinkBERT?
- michiyasunaga/LinkBERT has 455 stars on GitHub.
- Where can I find LinkBERT?
- michiyasunaga/LinkBERT is on GitHub at https://github.com/michiyasunaga/LinkBERT.