SkywalkerDarren/chatWeb
A system that crawls web pages and documents, extracts text, stores embeddings in a vector database, and uses GPT-3.5 to answer questions based on retrieved content.
Not currently ranked — collecting fresh signals.
star history
ChatWeb crawls any webpage or extracts text from PDF, DOCX, and TXT files. It generates embedded summaries using GPT-3.5’s embedding API, stores vector-text mappings in a vector database (FAISS or pgvector), and retrieves the most similar text chunks via nearest neighbor search to generate answers. It improves accuracy by generating vectors from keywords rather than raw questions, effectively breaking through token limits by extracting relevant content from large texts.
Frequently asked
- What is SkywalkerDarren/chatWeb?
- A system that crawls web pages and documents, extracts text, stores embeddings in a vector database, and uses GPT-3.5 to answer questions based on retrieved content.
- Is chatWeb open source?
- Yes — SkywalkerDarren/chatWeb is open source, released under the MIT license.
- What language is chatWeb written in?
- SkywalkerDarren/chatWeb is primarily written in Python.
- How popular is chatWeb?
- SkywalkerDarren/chatWeb has 916 stars on GitHub.
- Where can I find chatWeb?
- SkywalkerDarren/chatWeb is on GitHub at https://github.com/SkywalkerDarren/chatWeb.