Commit Graph

1312 Commits

Author SHA1 Message Date
imClumsyPanda a87a2815da update knowledge_base and dialogue page of webui 2023-09-15 13:45:47 +08:00
imClumsyPanda eb49a91190 update webui_pages/knowledge_base.py 2023-09-15 11:04:23 +08:00
imClumsyPanda f066e427ce update webui_pages/knowledge_base.py 2023-09-15 10:58:52 +08:00
liunux4odoo 63f0d27167
在readme中添加支持的在线API模型 (#1485) 2023-09-15 10:16:15 +08:00
imClumsyPanda 20eef45a58 update README.md 2023-09-15 09:53:58 +08:00
liunux4odoo 30aeec1c45
修改智谱API默认名称,给WEBUI设置主题 (#1484)
* 給WEBUI设置主题

* chatglm-api更名为zhipu-api
2023-09-15 01:48:02 +08:00
liunux4odoo f0f1dc2537 补充更多千帆平台支持的模型;除了指定模型名称,支持直接指定模型APIURL,便于填写单独申请的模型地址 2023-09-15 00:30:18 +08:00
liunux4odoo 4cf2e5ea5e 修复文心一言,添加测试用例 2023-09-14 23:37:34 +08:00
imClumsyPanda fbaca1009e update requirements.txt, requirements_api.txt, test_different_splitter.py and chinese_recursive_text_splitter.py 2023-09-14 22:59:05 +08:00
imClumsyPanda 1813814a65 update model_config.py.example 2023-09-14 22:29:49 +08:00
imClumsyPanda 4c61a642cf update model_config.py.example 2023-09-14 22:29:26 +08:00
imClumsyPanda bda1b01a2d update model_config.py.example 2023-09-14 22:27:57 +08:00
imClumsyPanda 17985c7f8e update splitter.md 2023-09-14 22:24:58 +08:00
Zhi-guo Huang 3745b1d1bb
Merge pull request #1479 from hzg0601/dev
update readme,readme_en:更新fastchat信息;删除订阅号二维码
2023-09-14 17:01:35 +08:00
hzg0601 6b734ced56 update readme,readme_en:更新fastchat信息;删除订阅号二维码 2023-09-14 17:00:20 +08:00
imClumsyPanda 883445e5b2
增加 百度文心一言 API 接入支持 (#1477)
* add RapidOCRPDFLoader

* update mypdfloader.py and requirements.txt

* add myimgloader.py

* add test samples

* add TODO to mypdfloader

* add loaders to KnowledgeFile class

* add loaders to KnowledgeFile class

* add ChineseRecursiveTextSplitter

* add ChineseRecursiveTextSplitter

* add baidu ernie api
2023-09-14 15:30:06 +08:00
zR 769d75d784
修复测试文件 (#1467)
Co-authored-by: zR <zRzRzRzRzRzRzR>
2023-09-13 17:12:05 +08:00
imClumsyPanda dc413120e2 update text splitter unit test name 2023-09-13 15:53:07 +08:00
imClumsyPanda f75812afcb update text splitter unit test name 2023-09-13 15:47:34 +08:00
zR bfdbe69fa1
增加了自定义分词器适配 (#1462)
* 添加了自定义分词器适配和测试文件
---------

Co-authored-by: zR <zRzRzRzRzRzRzR>
2023-09-13 15:42:12 +08:00
liunux4odoo c4cb4e19e5
清理不必要的依赖,增加星火API需要的websockets (#1463) 2023-09-13 15:35:04 +08:00
liunux4odoo 99b862dfc8 fix SparkApi import error 2023-09-13 14:03:09 +08:00
liunux4odoo 512638a3b1
支持讯飞星火大模型在线API (#1460)
* 支持讯飞星火大模型在线API
2023-09-13 13:51:05 +08:00
liunux4odoo 16d8809c9a
api和webui知识库操作支持chunk_size/overlap_size/zh_title_enhance参数 (#1459) 2023-09-13 11:19:47 +08:00
liunux4odoo 9defa4332e fix minimax api history error 2023-09-13 10:12:31 +08:00
liunux4odoo 8b040620de
对话接口支持temperature参数 (#1455) 2023-09-13 10:00:54 +08:00
liunux4odoo a03b8d330d
升级到langchain==0.0.287,fschat==0.0.28;处理langchain.Milvus bug(#10492) (#1454)
* 修复milvus_kb_service中一些bug,添加文档后将数据同步到数据库
* 升级到langchain==0.0.287,fschat==0.0.28;处理langchain.Milvus bug(#10492)
* 修复切换模型BUG: 从在线API切换模型时出错
2023-09-13 08:43:11 +08:00
liunux4odoo efd6d4a251
修复milvus_kb_service中一些bug,添加文档后将数据同步到数据库 (#1452) 2023-09-12 22:34:03 +08:00
imClumsyPanda 4aa14b859e
增加 ChineseRecursiveTextSplitter (#1447)
* add RapidOCRPDFLoader

* update mypdfloader.py and requirements.txt

* add myimgloader.py

* add test samples

* add TODO to mypdfloader

* add loaders to KnowledgeFile class

* add loaders to KnowledgeFile class

* add ChineseRecursiveTextSplitter

* add ChineseRecursiveTextSplitter
2023-09-12 17:38:52 +08:00
imClumsyPanda 2020f5bb88 merge master 2023-09-12 15:51:28 +08:00
liunux4odoo 03c021d50f
添加MiniMax在线API支持 (#1445) 2023-09-12 15:24:47 +08:00
imClumsyPanda 8064586374 delete .doc from supported formats 2023-09-12 15:22:18 +08:00
liunux4odoo 8a12b9c658
添加自动关闭issue的action (#1437) 2023-09-12 10:16:19 +08:00
liunux4odoo 60f88ae92e
fix: 由于server.utils导入model_workers时,导入fastchat model_worker,导致startup启动延迟明显 (#1436) 2023-09-12 08:52:00 +08:00
liunux4odoo 6a03611750
fix api and webui: (#1435)
1. fix #1431: 优化知识库问答与搜索引擎问答的API接口,避免docs重复返回
2. startup.py根据configs.log_verbose控制log级别
3. 修复/llm_model/list_models的bug:
   只有一个参数时,fastapi未返回json导致视图函数出错
2023-09-12 08:31:17 +08:00
liunux4odoo 22ff073309
修改Embeddings和FAISS缓存加载方式,知识库相关API接口支持多线程并发 (#1434)
* 修改Embeddings和FAISS缓存加载方式,支持多线程,支持内存FAISS

* 知识库相关API接口支持多线程并发

* 根据新的API接口调整ApiRequest和测试用例

* 删除webui.py失效的启动说明
2023-09-11 20:41:41 +08:00
imClumsyPanda 518d3c4951
Update README.md 2023-09-11 15:18:03 +08:00
imClumsyPanda be621772ab
Add files via upload 2023-09-11 15:17:38 +08:00
Zhi-guo Huang d0e654d847
Merge pull request #1422 from hzg0601/dev
update readme.md,readme_cn.md:更新订阅号QRcode
2023-09-09 17:36:10 +08:00
hzg0601 ed6df7cd24 update readme.md,readme_cn.md:更新订阅号QRcode 2023-09-09 17:34:56 +08:00
glide-the b0a9d8f30e 增加显示ocr识别进度 2023-09-08 21:32:41 +08:00
glide-the 60e6887e94
Merge pull request #1419 from glide-the/dev
增加显示详细日志开关
2023-09-08 20:50:31 +08:00
glide-the 5550e3c1a9 增加显示详细日志开关 2023-09-08 20:48:31 +08:00
liunux4odoo 228fc5bd93 fix startup.py: correct processes count 2023-09-08 15:24:43 +08:00
liunux4odoo f082edbaad fix startup.py: correct processes count 2023-09-08 15:23:54 +08:00
liunux4odoo f94f2793f8
update startup.py: (#1416)
1. 恢复模型切换功能
2. --model-name支持多个名称(空格分开),同时启动多个模型
3. 优化服务启动顺序。严格按照顺序启动:controller -> [openai-api,
   model_worker, api_worker]并行 -> api.py -> webui.py
4. 修复:从在线API模型切换到本地模型时失败
2023-09-08 15:18:13 +08:00
Kyle Zhang 6e9a7e0c89
chore: update .gitignore (#1364)
Added '/' before knowledge_base/ and configs/*.py to make them match
directories at root folder only.

Signed-off-by: Kyle Zhang <zhangk1985@gmail.com>
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-09-08 13:29:40 +08:00
liunux4odoo 775870a516
改变api视图函数的sync/async,提高api并发能力: (#1414)
1. 4个chat类接口改为async
2. 知识库操作,涉及向量库修改的使用async,避免FAISS写入错误;涉及向量库读取的改为sync,提高并发
2023-09-08 12:25:02 +08:00
roki1031 196bde3ead
提醒fitz包是来自pyMuPDF (#1407) 2023-09-08 12:20:20 +08:00
liunux4odoo 1195eb75eb merge pr1413 2023-09-08 10:30:07 +08:00