diff --git a/README.md b/README.md index 1546847..3f32051 100644 --- a/README.md +++ b/README.md @@ -14,9 +14,16 @@ **[2023/04/07]** 1. 解决加载 ChatGLM 模型时发生显存占用为双倍的问题 (感谢 [@suc16](https://github.com/suc16) 和 [@myml](https://github.com/myml)) ; -2. 新增清理显存机制。 +2. 新增清理显存机制; 3. 新增`nghuyong/ernie-3.0-nano-zh`和`nghuyong/ernie-3.0-base-zh`作为 Embedding 模型备选项,相比`GanymedeNil/text2vec-large-chinese`占用显存资源更少 (感谢 [@lastrei](https://github.com/lastrei))。 +**[2023/04/09]** +1. 使用`langchain`中的`RetrievalQA`替代之前选用的`ChatVectorDBChain`,替换后可以有效减少提问 2-3 次后因显存不足而停止运行的问题; +2. 在`knowledge_based_chatglm.py`中增加`EMBEDDING_MODEL`、`VECTOR_SEARCH_TOP_K`、`LLM_MODEL`、`LLM_HISTORY_LEN`、`REPLY_WITH_SOURCE`参数值设置; +3. 增加 GPU 显存需求更小的`chatglm-6b-int4`、`chatglm-6b-int4-qe`作为 LLM 模型备选项; +4. 更正`README.md`中的代码错误(感谢 [@calcitem](https://github.com/calcitem))。 + + ## 使用方式 ### 硬件需求 @@ -31,7 +38,8 @@ - Embedding 模型硬件需求 本项目中默认选用的 Embedding 模型 [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese/tree/main) 约占用显存 3GB,也可修改为在 CPU 中运行。 - +### 软件需求 +本项目已在 python 3.8 环境下完成测试。 ### 1. 安装 python 依赖包 ```commandline pip install -r requirements.txt diff --git a/README_en.md b/README_en.md index 7403b91..3a211d1 100644 --- a/README_en.md +++ b/README_en.md @@ -16,6 +16,12 @@ 2. Add gpu memory clear function after each call of ChatGLM. 3. Add `nghuyong/ernie-3.0-nano-zh` and `nghuyong/ernie-3.0-base-zh` as Embedding model alternatives,costing less gpu than `GanymedeNil/text2vec-large-chinese` (Thanks to [@lastrei](https://github.com/lastrei)) +**[2023/04/09]** +1. Using `RetrievalQA` in `langchain` to replace the previously selected `ChatVectorDBChain`, the replacement can effectively solve the problem of program stopping after 2-3 questions due to insufficient gpu memory. +2. Add `EMBEDDING_MODEL`, `VECTOR_SEARCH_TOP_K`, `LLM_MODEL`, `LLM_HISTORY_LEN`, `REPLY_WITH_SOURCE` parameter value settings in `knowledge_based_chatglm.py`. +3. Add `chatglm-6b-int4`, `chatglm-6b-int4-qe` with smaller GPU memory requirements as LLM model alternatives. +4. Correct code errors in `README.md` (Thanks to [@calcitem](https://github.com/calcitem)). + ## Usage ### Hardware Requirements @@ -31,6 +37,8 @@ The default Embedding model in this repo is [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese/tree/main), 3GB GPU Memory required when running on GPU. +### Software Requirements +This repo has been tested in python 3.8 environment。 ### 1. install python packages ```commandline