Commit Graph

59 Commits

Author SHA1 Message Date
imClumsyPanda 11dd2b5b84 update api.py 2023-06-14 00:35:33 +08:00
imClumsyPanda a887df1715 add knowledge_base folder and move vector_store and content inside 2023-06-13 23:54:29 +08:00
imClumsyPanda 7863e0fea8 updata MyFAISS 2023-06-12 00:06:06 +08:00
imClumsyPanda f1f742ce44 add self-defined class MyFAISS 2023-06-07 23:18:47 +08:00
imClumsyPanda 8b7c2e417c add support for csv file 2023-06-07 22:14:08 +08:00
shrimp 57b4f9306d
修复BUG (#570)
一、修复模型历史记录仅一条问题,修改了模型加载代码,已修复。 
二、修复模型历史记录仅一条问题,修改了webui有一个地方有问题,已修复。
三、知识库单条数据入库因知识库名称缓存问题导致的BUG,也已修复。
2023-06-07 18:12:51 +08:00
imClumsyPanda b1ba2003be update webui.py 2023-05-31 20:55:45 +08:00
liunux4odoo 88fee48d8c
use lru_cache to optimize loading of local vector store (#496)
* Add files via upload

* Update README.md

* use lru_cache to optimize loading of local vector store

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
Co-authored-by: liunux <liunux@office>
2023-05-30 23:05:39 +08:00
Ding Junyao f7e120fe56
add tree func for reading files in dirs (#471) 2023-05-27 22:00:43 +08:00
glide-the c4ee36b8ac 删除 AnswerResultStream 、generate_with_callback收集器 2023-05-25 21:07:40 +08:00
imClumsyPanda bca32cae83 update local_doc_qa.py 2023-05-22 18:09:23 +08:00
imClumsyPanda c79058f064
更新 local_doc_qa.py 2023-05-22 13:14:23 +08:00
yawudede 4634a578f4
local_doc_qa.py 2023-05-22 10:39:55 +08:00
yawudede 9140a5e70c
local_doc_qa.py 2023-05-22 10:36:41 +08:00
yawudede 037b5e4502
local_doc_qa.py
添加精排
2023-05-22 10:32:58 +08:00
imClumsyPanda e352a04cbb merge master 2023-05-22 00:18:48 +08:00
imClumsyPanda 9c422cc6bc update bing_search.py 2023-05-21 22:08:38 +08:00
imClumsyPanda 2681728329 update local_doc_qa.py 2023-05-20 01:54:08 +08:00
imClumsyPanda e8a37ff4c7 update loader.py 2023-05-20 01:24:35 +08:00
zhenkaivip 9f5d1afc31
fix bug : 1、历史最后一条丢失;2、上下两行连读造成歧义 (#415)
* fix _call bug

* fix

* fix bug : 上下两行连读造成歧义
2023-05-20 01:19:22 +08:00
glide-the 33bbb4779e llm_model_dict 处理了loader的一些预设行为,如加载位置,模型名称,模型处理器实例, 定义checkpoint名称和远程路径
loader.py: 模型重载
定义 generatorAnswer 增加 AnswerResultStream
   定义generate_with_callback收集器,在每次响应时将队列数据同步到AnswerResult
requirements.txt 变更项目依赖
2023-05-18 22:54:41 +08:00
imClumsyPanda 621a0fe686 update cli_demo.py 2023-05-13 11:45:57 +08:00
zhenkaivip dd93837343
使用paddleocr实现 (#342)
* jpg and png ocr

* fix

* write docs to tmp file

* fix

* [BUGFIX] local_doc_qa.py line 172: logging have no end args. (#323)

* image loader

* fix

* fix

* update api.py

* update api.py

* update api.py

* update README.md

* update api.py

* add pdf_loader

* fix

---------

Co-authored-by: RainGather <3255329+RainGather@users.noreply.github.com>
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-13 08:45:17 +08:00
Tyler Luan 64406cee45
Add MOSS (#317)
* Add MOSS

* Update local_doc_qa.py

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-11 18:42:19 +08:00
imClumsyPanda 5852b4c62e update pdf read strategy 2023-05-11 10:54:45 +08:00
imClumsyPanda fbdc62d910 merge master 2023-05-10 17:59:28 +08:00
shrimp 55504fcd4d
新增加知识库测试能力 (#302) 2023-05-10 17:18:20 +08:00
JohnSaxon 466bfb7a3f
logger bug introduced by https://github.com/imClumsyPanda/langchain-ChatGLM/pull/268 , fixed #295 (#297) 2023-05-10 13:28:44 +08:00
imClumsyPanda c613c41d0e update local_doc_qa.py 2023-05-08 23:49:57 +08:00
imClumsyPanda e2d7452c7b update chinese_text_splitter.py 2023-05-06 23:26:49 +08:00
keenzhu 2c72a00954
Update local_doc_qa.py
增加命令行导入的进度条显示
增加文本提示
2023-05-06 18:39:58 +08:00
shrimp 14d998b8e6
可选择lora权重加载 (#231)
* Add files via upload

增加lora权重使用

* Update model_config.py

* Add files via upload

修复一个小错误,少写了模型加载

* 使用lora微调的权重

使用lora微调的权重

* Update model_config.py
2023-05-04 21:01:55 +08:00
Winter 47922d2ee3
fix: 扩展文档的代码逻辑 (#227)
* fix: 扩展文档的代码逻辑

* Update local_doc_qa.py

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-04 20:58:15 +08:00
imClumsyPanda 07ff81a119 update torch_gc 2023-05-04 20:48:36 +08:00
imClumsyPanda 2c1fd2bdd5 add api.py 2023-05-03 22:31:28 +08:00
imClumsyPanda a4e67a67b4 update local_doc_qa.py 2023-05-02 01:11:05 +08:00
imClumsyPanda 4df9d76f8a add streaming option in configs/model_config.py 2023-05-01 23:52:28 +08:00
imClumsyPanda 0e8cc0d16c add streaming option in configs/model_config.py 2023-05-01 23:52:28 +08:00
imClumsyPanda 98a8281b29 update text_splitter 2023-04-28 00:02:42 +08:00
imClumsyPanda 966def8cfe add stream support to webui.py 2023-04-26 23:27:15 +08:00
imClumsyPanda b4aefca555 add stream support to cli_demo.py 2023-04-26 23:27:15 +08:00
imClumsyPanda 88ab9a1d21 update webui.py and local_doc_qa.py 2023-04-26 23:27:15 +08:00
wangxinkai 059fe82887 feat: 添加mmr相似度搜索,支持返回相似度分数 2023-04-26 23:27:15 +08:00
imClumsyPanda a1033698dd update local_doc_qa.py, webui.py and README.md 2023-04-26 23:27:15 +08:00
imClumsyPanda 0706a29057 update embedding define in local_doc_qa.py and langchain version in requirements.txt 2023-04-22 12:30:27 +08:00
imClumsyPanda b9d8824fa5 Update local_doc_qa.py 2023-04-21 21:27:20 +08:00
imClumsyPanda ec677beed0 Update local_doc_qa.py 2023-04-21 21:27:20 +08:00
soon 37ceeae6e2 提交更 2023-04-21 21:27:20 +08:00
imClumsyPanda f147043253 fix bug in chatglm_llm.py 2023-04-20 12:50:51 +08:00
imClumsyPanda 7d4560e599 update local_doc_qa.py 2023-04-19 21:29:20 +08:00