7291e77978
support for bge-large-zh-noinstruct and openai text-embedding-ada-002 ( #1119 )
...
* support for bge-large-zh-noinstruct and openai text-embedding-ada-002
* 完善了readme,修改了BGE模型的载入方式
* 补充了readme中bge-large-zh-noinstruct的信息
* Update faiss_kb_service.py
* Update utils.py
---------
Co-authored-by: zR <zRzRzRzRzRzRzR>
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-08-23 22:55:53 +08:00
f40bb69224
优化FAISS向量库多文件操作；修复recreate_vector_store，大量文件时不再超时。
2023-08-20 19:10:29 +08:00
150a78bfd9
update kb_doc_api:make faiss cache working; delete vector store docs before add duplicate docs
2023-08-20 16:52:49 +08:00
67b8ebef52
update api and webui:
...
1. 增加search_docs接口，返回原始知识库检索文档，close #1103
2. 为FAISS检索增加score_threshold参数。milvus和PG暂不支持
2023-08-16 13:18:58 +08:00
8df00d0b04
add SCORE_THRESHOLD to faiss and milvus kb service
2023-08-10 00:36:59 +08:00
1b70fb5f9b
update faiss_kb_service.py
2023-08-09 22:57:36 +08:00
c7b91bfaf1
update api/kb_doc_api and knowledge base management ui:
...
1. add update_doc to api which can udpate vector store from existed
content file
2. add parameter `delete_content` to delete_doc api. user can decide
whether delete local content file when delete doc.
3. fix bug in ApiReqeust.upload_doc
4. support listing docs existed in local folder bu not in db
2023-08-09 16:52:04 +08:00
b98f5fd0b9
update knowledge base kbservice and api:
...
1. make huggingfaceembeddings hashable
2. unify embeddings' loading method for all kbservie
3. make ApiRequest skip empty content when streaming json to avoid dict
KeyError
2023-08-09 10:46:01 +08:00
584e3a9234
remove server/knowledge_base/knowledge_base_factory.py
2023-08-08 16:21:00 +08:00
44c713ef98
use KBServiceFactory to replace all the KnowledgeBase.
...
make KBServiceFactory support embed_model parameter.
rewrite api: recreate_vector_store.
fix some bugs.
2023-08-07 20:37:16 +08:00
0f46185cfb
添加Milvus库 ( #1011 )
2023-08-07 16:56:57 +08:00
18d31f5116
add KBService and KBServiceFactory class
2023-08-06 23:44:02 +08:00
zR
liunux4odoo
imClumsyPanda
zqt996