weiweiw
/
Langchain-Chatchat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Dev streaming oom (#722)

This commit is contained in:
allseeeeeee 2023-06-26 19:12:50 +08:00 committed by GitHub
parent 623d3d44ec
commit 521d9e2f98
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@ -198,6 +198,7 @@ Web UI 可以实现如下功能：
- [ ] 知识图谱/图数据库接入 - [ ] 知识图谱/图数据库接入
- [ ] Agent 实现 - [ ] Agent 实现
- [x] 增加更多 LLM 模型支持 - [x] 增加更多 LLM 模型支持
- [x] [THUDM/chatglm2-6b](https://huggingface.co/THUDM/chatglm2-6b)
- [x] [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b) - [x] [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)
- [x] [THUDM/chatglm-6b-int8](https://huggingface.co/THUDM/chatglm-6b-int8) - [x] [THUDM/chatglm-6b-int8](https://huggingface.co/THUDM/chatglm-6b-int8)
- [x] [THUDM/chatglm-6b-int4](https://huggingface.co/THUDM/chatglm-6b-int4) - [x] [THUDM/chatglm-6b-int4](https://huggingface.co/THUDM/chatglm-6b-int4)
@ -229,7 +230,7 @@ Web UI 可以实现如下功能：
- [x] VUE 前端 - [x] VUE 前端
## 项目交流群 ## 项目交流群
<img src="img/qr_code_35.jpg" alt="二维码" width="300" height="300" /> <img src="img/qr_code_36.jpg" alt="二维码" width="300" height="300" />
🎉 langchain-ChatGLM 项目微信交流群，如果你也对本项目感兴趣，欢迎加入群聊参与讨论交流。 🎉 langchain-ChatGLM 项目微信交流群，如果你也对本项目感兴趣，欢迎加入群聊参与讨论交流。

BIN
img/qr_code_35.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 193 KiB

BIN
img/qr_code_36.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 247 KiB

1
models/chatglm_llm.py
View File

@ -65,6 +65,7 @@ class ChatGLM(BaseAnswer, LLM, ABC):
answer_result.history = history answer_result.history = history
answer_result.llm_output = {"answer": stream_resp} answer_result.llm_output = {"answer": stream_resp}
yield answer_result yield answer_result
self.checkPoint.clear_torch_cache()
else: else:
response, _ = self.checkPoint.model.chat( response, _ = self.checkPoint.model.chat(
self.checkPoint.tokenizer, self.checkPoint.tokenizer,