weiweiw
/
Langchain-Chatchat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
27 Commits 1 Branch 0 Tags 72 MiB
JavaScript 69.2%
Python 30.8%
Go to file
imClumsyPanda 9bb7c219ff
Merge pull request #28 from lastrei/master 
根据4.7日更新代码，解决冲突
 		2023-04-07 12:22:22 +08:00
LICENSE Create LICENSE 2023-04-07 11:41:10 +08:00
README.md update README.md 2023-04-07 11:15:03 +08:00
README_en.md add torch_gc to clear gpu cache in knowledge_based_chatglm.py 2023-04-07 11:02:23 +08:00
chatglm_llm.py add torch_gc to clear gpu cache in knowledge_based_chatglm.py 2023-04-07 10:46:02 +08:00
knowledge_based_chatglm.py 根据4.7日更新代码，解决冲突 2023-04-07 12:01:15 +08:00
requirements.txt Update requirements.txt 2023-04-05 21:39:42 +08:00

README_en.md

ChatGLM Application Based on Local Knowledge

Introduction

🌍 中文文档

🤖️ A local knowledge based LLM Application with ChatGLM-6B and langchain.

💡 Inspired by document.ai by GanymedeNil and ChatGLM-6B Pull Request by AlexZhangji.

✅ In this project, GanymedeNil/text2vec-large-chinese is used as Embedding Model，and ChatGLM-6B used as LLM。Based on those models，this project can be deployed offline with all open source models。

Update

[2023/04/07]

  1. Fix bug which costs twice gpu memory (Thanks to @suc16 and @myml).
  2. Add gpu memory clear function after each call of ChatGLM.

Usage

Hardware Requirements

  • ChatGLM Hardware Requirements

    Quantization Level GPU Memory
    FP16（no quantization） 13 GB
    INT8 10 GB
    INT4 6 GB

  • Embedding Hardware Requirements

    The default Embedding model in this repo is GanymedeNil/text2vec-large-chinese, 3GB GPU Memory required when running on GPU.

1. install python packages

pip install -r requirements

Attention: With langchain.document_loaders.UnstructuredFileLoader used to connect with local knowledge file, you may need some other dependencies as mentioned in langchain documentation

2. Run knowledge_based_chatglm.py script

python knowledge_based_chatglm.py

Roadmap

  • local knowledge based application with langchain + ChatGLM-6B
  • unstructured files loaded with langchain
  • more different file format loaded with langchain
  • implement web ui DEMO with gradio/streamlit
  • implement API with fastapi，and web ui DEMO with API