11dd2b5b84
update api.py
2023-06-14 00:35:33 +08:00
a887df1715
add knowledge_base folder and move vector_store and content inside
2023-06-13 23:54:29 +08:00
7863e0fea8
updata MyFAISS
2023-06-12 00:06:06 +08:00
f1f742ce44
add self-defined class MyFAISS
2023-06-07 23:18:47 +08:00
8b7c2e417c
add support for csv file
2023-06-07 22:14:08 +08:00
57b4f9306d
修复BUG ( #570 )
...
一、修复模型历史记录仅一条问题，修改了模型加载代码，已修复。
二、修复模型历史记录仅一条问题，修改了webui有一个地方有问题，已修复。
三、知识库单条数据入库因知识库名称缓存问题导致的BUG，也已修复。
2023-06-07 18:12:51 +08:00
b1ba2003be
update webui.py
2023-05-31 20:55:45 +08:00
88fee48d8c
use lru_cache to optimize loading of local vector store ( #496 )
...
* Add files via upload
* Update README.md
* use lru_cache to optimize loading of local vector store
---------
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
Co-authored-by: liunux <liunux@office>
2023-05-30 23:05:39 +08:00
f7e120fe56
add tree func for reading files in dirs ( #471 )
2023-05-27 22:00:43 +08:00
5f010823aa
Build an abstract dialogue model using classes and methods to represent different dialogue elements
...
Fixes # None
Before submitting
If you want to review, please refer to the quick start example provided in langchain/chains/dialogue_answering/main.py. You may need to set the openaikey and the following startup parameters: --dialogue-path: the location of the dialogue file, --embedding-model: the HuggingFaceEmbeddings model to use (defaults to GanymedeNil/text2vec-large-chinese) if not specified.
Regarding the format of the dialogue file, please refer to the following information:
```text
sun:
Has the offline model been run?
glide-the:
Yes, it has been run, but the results are not very satisfactory.
glide-the:
It lacks chat intelligence and falls far behind in terms of logic and reasoning.
sun:
Are you available for voice chat?
glide-the:
I'm considering using this offline model: https://huggingface.co/chat
glide-the:
voice chat okay.
glide-the:
You can take a look at the dev_agent branch of the langchain-chatglm project.
glide-the:
There's a dialogue model question-answering example under the agent.
sun:
Alright.
glide-the:
The specified chat record file is exported from WeChat.
```
2023-05-26 01:49:07 +08:00
c4ee36b8ac
删除 AnswerResultStream 、generate_with_callback收集器
2023-05-25 21:07:40 +08:00
bca32cae83
update local_doc_qa.py
2023-05-22 18:09:23 +08:00
c79058f064
更新 local_doc_qa.py
2023-05-22 13:14:23 +08:00
4634a578f4
local_doc_qa.py
2023-05-22 10:39:55 +08:00
9140a5e70c
local_doc_qa.py
2023-05-22 10:36:41 +08:00
037b5e4502
local_doc_qa.py
...
添加精排
2023-05-22 10:32:58 +08:00
e352a04cbb
merge master
2023-05-22 00:18:48 +08:00
9c422cc6bc
update bing_search.py
2023-05-21 22:08:38 +08:00
2681728329
update local_doc_qa.py
2023-05-20 01:54:08 +08:00
e8a37ff4c7
update loader.py
2023-05-20 01:24:35 +08:00
9f5d1afc31
fix bug : 1、历史最后一条丢失；2、上下两行连读造成歧义 ( #415 )
...
* fix _call bug
* fix
* fix bug : 上下两行连读造成歧义
2023-05-20 01:19:22 +08:00
33bbb4779e
llm_model_dict 处理了loader的一些预设行为，如加载位置，模型名称，模型处理器实例, 定义checkpoint名称和远程路径
...
loader.py: 模型重载
定义 generatorAnswer 增加 AnswerResultStream
定义generate_with_callback收集器，在每次响应时将队列数据同步到AnswerResult
requirements.txt 变更项目依赖
2023-05-18 22:54:41 +08:00
621a0fe686
update cli_demo.py
2023-05-13 11:45:57 +08:00
dd93837343
使用paddleocr实现 ( #342 )
...
* jpg and png ocr
* fix
* write docs to tmp file
* fix
* [BUGFIX] local_doc_qa.py line 172: logging have no end args. (#323 )
* image loader
* fix
* fix
* update api.py
* update api.py
* update api.py
* update README.md
* update api.py
* add pdf_loader
* fix
---------
Co-authored-by: RainGather <3255329+RainGather@users.noreply.github.com>
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-13 08:45:17 +08:00
64406cee45
Add MOSS ( #317 )
...
* Add MOSS
* Update local_doc_qa.py
---------
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-11 18:42:19 +08:00
5852b4c62e
update pdf read strategy
2023-05-11 10:54:45 +08:00
fbdc62d910
merge master
2023-05-10 17:59:28 +08:00
55504fcd4d
新增加知识库测试能力 ( #302 )
2023-05-10 17:18:20 +08:00
466bfb7a3f
logger bug introduced by https://github.com/imClumsyPanda/langchain-ChatGLM/pull/268 , fixed #295 ( #297 )
2023-05-10 13:28:44 +08:00
c613c41d0e
update local_doc_qa.py
2023-05-08 23:49:57 +08:00
e2d7452c7b
update chinese_text_splitter.py
2023-05-06 23:26:49 +08:00
2c72a00954
Update local_doc_qa.py
...
增加命令行导入的进度条显示
增加文本提示
2023-05-06 18:39:58 +08:00
14d998b8e6
可选择lora权重加载 ( #231 )
...
* Add files via upload
增加lora权重使用
* Update model_config.py
* Add files via upload
修复一个小错误，少写了模型加载
* 使用lora微调的权重
使用lora微调的权重
* Update model_config.py
2023-05-04 21:01:55 +08:00
47922d2ee3
fix: 扩展文档的代码逻辑 ( #227 )
...
* fix: 扩展文档的代码逻辑
* Update local_doc_qa.py
---------
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-04 20:58:15 +08:00
07ff81a119
update torch_gc
2023-05-04 20:48:36 +08:00
2c1fd2bdd5
add api.py
2023-05-03 22:31:28 +08:00
a4e67a67b4
update local_doc_qa.py
2023-05-02 01:11:05 +08:00
4df9d76f8a
add streaming option in configs/model_config.py
2023-05-01 23:52:28 +08:00
0e8cc0d16c
add streaming option in configs/model_config.py
2023-05-01 23:52:28 +08:00
98a8281b29
update text_splitter
2023-04-28 00:02:42 +08:00
966def8cfe
add stream support to webui.py
2023-04-26 23:27:15 +08:00
b4aefca555
add stream support to cli_demo.py
2023-04-26 23:27:15 +08:00
88ab9a1d21
update webui.py and local_doc_qa.py
2023-04-26 23:27:15 +08:00
daafe8d5fa
feat: 重写FAISS和HuggingFaceEmbedding，采用MMR相似向量检索
2023-04-26 23:27:15 +08:00
059fe82887
feat: 添加mmr相似度搜索，支持返回相似度分数
2023-04-26 23:27:15 +08:00
a1033698dd
update local_doc_qa.py, webui.py and README.md
2023-04-26 23:27:15 +08:00
0706a29057
update embedding define in local_doc_qa.py and langchain version in requirements.txt
2023-04-22 12:30:27 +08:00
b9d8824fa5
Update local_doc_qa.py
2023-04-21 21:27:20 +08:00
ec677beed0
Update local_doc_qa.py
2023-04-21 21:27:20 +08:00
37ceeae6e2
提交更
2023-04-21 21:27:20 +08:00
imClumsyPanda
shrimp
liunux4odoo
Ding Junyao
glide-the
yawudede
zhenkaivip
Tyler Luan
JohnSaxon
keenzhu
Winter
wangxinkai
soon