weiweiw
/
Langchain-Chatchat
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

GLM3支持传入提示词 (#2058) 

支持传入提示词的Agent ChatGLM3-6B
This commit is contained in:
zR 2023-11-14 17:45:22 +08:00 committed by GitHub
parent e78a804ec6
commit 21b079d751
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 155 additions and 160 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

204
configs/prompt_config.py.example
View File

@ -17,96 +17,118 @@
# - input: 用户输入内容
# - agent_scratchpad: Agent的思维记录
PROMPT_TEMPLATES = {
"completion": {
"default": "{input}"
},
PROMPT_TEMPLATES = {}
"llm_chat": {
"default": "{{ input }}",
"py":
"""
你是一个聪明的代码助手，请你给我写出简单的py代码。 \n
{{ input }}
"""
,
},
"knowledge_base_chat": {
"default":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，不允许在答案中添加编造成分，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"text":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"Empty": # 搜不到内容的时候调用，此时没有已知信息，这个Empty可以更改，但不能删除，会影响程序使用
"""
<指令>请根据用户的问题，进行简洁明了的回答</指令>
<问题>{{ question }}</问题>
""",
},
"search_engine_chat": {
"default":
"""
<指令>这是我搜索到的互联网信息，请你根据这些信息进行提取并有调理，简洁的回答问题。如果无法从中得到答案，请说 “无法搜索到能回答问题的内容”。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"search":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"Empty": # 搜不到内容的时候调用，此时没有已知信息，这个Empty可以更改，但不能删除，会影响程序使用
"""
<指令>请根据用户的问题，进行简洁明了的回答</指令>
<问题>{{ question }}</问题>
""",
},
"agent_chat": {
"default":
"""
Answer the following questions as best you can. If it is in order, you can use some tools appropriately.You have access to the following tools:
{tools}
Please note that the "知识库查询工具" is information about the "西交利物浦大学" ,and if a question is asked about it, you must answer with the knowledge base，
Please note that the "天气查询工具" can only be used once since Question begin.
Use the following format:
Question: the input question you must answer1
Thought: you should always think about what to do and what tools to use.
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
history:
{history}
Question: {input}
Thought: {agent_scratchpad}
""",
"ChatGLM3": # ChatGLM3必须用官方的提示词，没有修改空间，目前参数都不会传入进去
"""
history:
{history}
Question: {input}
Thought: {agent_scratchpad}
""",
},
PROMPT_TEMPLATES["llm_chat"] = {
"default": "{{ input }}",
"py":
"""
你是一个聪明的代码助手，请你给我写出简单的py代码。 \n
{{ input }}
""",
}
PROMPT_TEMPLATES["knowledge_base_chat"] = {
"default":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，不允许在答案中添加编造成分，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"text":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
"Empty": # 搜不到知识库的时候使用
"""
请你回答我的问题:
{{ question }}
\n
""",
}
PROMPT_TEMPLATES["search_engine_chat"] = {
"default":
"""
<指令>这是我搜索到的互联网信息，请你根据这些信息进行提取并有调理，简洁的回答问题。如果无法从中得到答案，请说 “无法搜索到能回答问题的内容”。 </指令>
<已知信息>{{ context }}</已知信息>
<问题>{{ question }}</问题>
""",
"search":
"""
<指令>根据已知信息，简洁和专业的来回答问题。如果无法从中得到答案，请说 “根据已知信息无法回答该问题”，答案请使用中文。 </指令>
<已知信息>{{ context }}</已知信息>、
<问题>{{ question }}</问题>
""",
}
PROMPT_TEMPLATES["agent_chat"] = {
"default":
"""
Answer the following questions as best you can. If it is in order, you can use some tools appropriately.You have access to the following tools:
{tools}
Please note that the "知识库查询工具" is information about the "西交利物浦大学" ,and if a question is asked about it, you must answer with the knowledge base，
Please note that the "天气查询工具" can only be used once since Question begin.
Use the following format:
Question: the input question you must answer1
Thought: you should always think about what to do and what tools to use.
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
history: {history}
Question: {input}
Thought: {agent_scratchpad}
""",
"ChatGLM3":
"""
You can answer using the tools, or answer directly using your knowledge without using the tools.Respond to the human as helpfully and accurately as possible.
You have access to the following tools:
{tools}
Use a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).
Valid "action" values: "Final Answer" or [{tool_names}]
Provide only ONE action per $JSON_BLOB, as shown:
```
{{{{
"action": $TOOL_NAME,
"action_input": $INPUT
}}}}
```
Follow this format:
Question: input question to answer
Thought: consider previous and subsequent steps
Action:
```
$JSON_BLOB
```
Observation: action result
... (repeat Thought/Action/Observation N times)
Thought: I know what to respond
Action:
```
{{{{
"action": "Final Answer",
"action_input": "Final response to human"
}}}}
Begin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation:.
history: {history}
Question: {input}
Thought: {agent_scratchpad}
""",
}

103
server/agent/custom_agent/ChatGLM3Agent.py
View File

@ -1,11 +1,11 @@
"""
This file is a modified version for ChatGLM3-6B the original ChatGLM3Agent.py file from the langchain repo.
"""
from __future__ import annotations
import yaml
from langchain.agents.structured_chat.output_parser import StructuredChatOutputParser
from langchain.memory import ConversationBufferWindowMemory
from typing import Any, List, Sequence, Tuple, Optional, Union
import os
from langchain.agents.agent import Agent
@ -13,7 +13,7 @@ from langchain.chains.llm import LLMChain
from langchain.prompts.chat import (
ChatPromptTemplate,
HumanMessagePromptTemplate,
SystemMessagePromptTemplate,
SystemMessagePromptTemplate, MessagesPlaceholder,
)
import json
import logging
@ -26,45 +26,6 @@ from langchain.callbacks.base import BaseCallbackManager
from langchain.schema.language_model import BaseLanguageModel
from langchain.tools.base import BaseTool
PREFIX = """
You can answer using the tools, or answer directly using your knowledge without using the tools.
Respond to the human as helpfully and accurately as possible.
You have access to the following tools:
"""
FORMAT_INSTRUCTIONS = """Use a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).
Valid "action" values: "Final Answer" or {tool_names}
Provide only ONE action per $JSON_BLOB, as shown:
```
{{{{
"action": $TOOL_NAME,
"action_input": $INPUT
}}}}
```
Follow this format:
Question: input question to answer
Thought: consider previous and subsequent steps
Action:
```
$JSON_BLOB
```
Observation: action result
... (repeat Thought/Action/Observation N times)
Thought: I know what to respond
Action:
```
{{{{
"action": "Final Answer",
"action_input": "Final response to human"
}}}}
```"""
SUFFIX = """Begin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation:.
Thought:"""
HUMAN_MESSAGE_TEMPLATE = "{input}\n\n{agent_scratchpad}"
logger = logging.getLogger(__name__)
@ -77,9 +38,6 @@ class StructuredChatOutputParserWithRetries(AgentOutputParser):
output_fixing_parser: Optional[OutputFixingParser] = None
"""The output fixing parser to use."""
def get_format_instructions(self) -> str:
return FORMAT_INSTRUCTIONS
def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
special_tokens = ["Action:", "<|observation|>"]
first_index = min([text.find(token) if token in text else len(text) for token in special_tokens])
@ -112,6 +70,7 @@ Action:
return parsed_obj
except Exception as e:
raise OutputParserException(f"Could not parse LLM output: {text}") from e
@property
def _type(self) -> str:
return "structured_chat_ChatGLM3_6b_with_retries"
@ -168,47 +127,57 @@ class StructuredGLM3ChatAgent(Agent):
def create_prompt(
cls,
tools: Sequence[BaseTool],
prefix: str = PREFIX,
suffix: str = SUFFIX,
human_message_template: str = HUMAN_MESSAGE_TEMPLATE,
format_instructions: str = FORMAT_INSTRUCTIONS,
prompt: str = None,
input_variables: Optional[List[str]] = None,
memory_prompts: Optional[List[BasePromptTemplate]] = None,
) -> BasePromptTemplate:
def tool_config_from_file(tool_name, directory="server/agent/tools/"):
"""search tool yaml and return json format"""
"""search tool yaml and return simplified json format"""
file_path = os.path.join(directory, f"{tool_name.lower()}.yaml")
try:
with open(file_path, 'r', encoding='utf-8') as file:
return yaml.safe_load(file)
tool_config = yaml.safe_load(file)
# Simplify the structure if needed
simplified_config = {
"name": tool_config.get("name", ""),
"description": tool_config.get("description", ""),
"parameters": tool_config.get("parameters", {})
}
return simplified_config
except FileNotFoundError:
print(f"File not found: {file_path}")
logger.error(f"File not found: {file_path}")
return None
except Exception as e:
print(f"An error occurred while reading {file_path}: {e}")
logger.error(f"An error occurred while reading {file_path}: {e}")
return None
tools_json = []
tool_names = ""
tool_names = []
for tool in tools:
tool_config = tool_config_from_file(tool.name)
if tool_config:
tools_json.append(tool_config)
tool_names.join(tool.name + ", ")
tool_names.append(tool.name)
# Format the tools for output
formatted_tools = "\n".join([
json.dumps(tool, ensure_ascii=False).replace("\"", "\\\"").replace("{", "{{").replace("}", "}}")
f"{tool['name']}: {tool['description']}, args: {tool['parameters']}"
for tool in tools_json
])
format_instructions = format_instructions.format(tool_names=tool_names)
template = "\n\n".join([prefix, formatted_tools, format_instructions, suffix])
formatted_tools = formatted_tools.replace("'", "\\'").replace("{", "{{").replace("}", "}}")
template = prompt.format(tool_names=tool_names,
tools=formatted_tools,
history="{history}",
input="{input}",
agent_scratchpad="{agent_scratchpad}")
if input_variables is None:
input_variables = ["input", "agent_scratchpad"]
_memory_prompts = memory_prompts or []
messages = [
SystemMessagePromptTemplate.from_template(template),
*_memory_prompts,
HumanMessagePromptTemplate.from_template(human_message_template),
]
return ChatPromptTemplate(input_variables=input_variables, messages=messages)
@ -217,12 +186,10 @@ class StructuredGLM3ChatAgent(Agent):
cls,
llm: BaseLanguageModel,
tools: Sequence[BaseTool],
prompt: str = None,
callback_manager: Optional[BaseCallbackManager] = None,
output_parser: Optional[AgentOutputParser] = None,
prefix: str = PREFIX,
suffix: str = SUFFIX,
human_message_template: str = HUMAN_MESSAGE_TEMPLATE,
format_instructions: str = FORMAT_INSTRUCTIONS,
input_variables: Optional[List[str]] = None,
memory_prompts: Optional[List[BasePromptTemplate]] = None,
**kwargs: Any,
@ -231,10 +198,7 @@ class StructuredGLM3ChatAgent(Agent):
cls._validate_tools(tools)
prompt = cls.create_prompt(
tools,
prefix=prefix,
suffix=suffix,
human_message_template=human_message_template,
format_instructions=format_instructions,
prompt=prompt,
input_variables=input_variables,
memory_prompts=memory_prompts,
)
@ -260,7 +224,9 @@ class StructuredGLM3ChatAgent(Agent):
def initialize_glm3_agent(
tools: Sequence[BaseTool],
llm: BaseLanguageModel,
prompt: str = None,
callback_manager: Optional[BaseCallbackManager] = None,
memory: Optional[ConversationBufferWindowMemory] = None,
agent_kwargs: Optional[dict] = None,
*,
tags: Optional[Sequence[str]] = None,
@ -269,12 +235,17 @@ def initialize_glm3_agent(
tags_ = list(tags) if tags else []
agent_kwargs = agent_kwargs or {}
agent_obj = StructuredGLM3ChatAgent.from_llm_and_tools(
llm, tools, callback_manager=callback_manager, **agent_kwargs
llm=llm,
tools=tools,
prompt=prompt,
callback_manager=callback_manager, **agent_kwargs
)
return AgentExecutor.from_agent_and_tools(
agent=agent_obj,
tools=tools,
callback_manager=callback_manager,
memory=memory,
tags=tags_,
**kwargs,
)

1
server/agent/custom_template.py
View File

@ -21,7 +21,6 @@ class CustomPromptTemplate(StringPromptTemplate):
kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
return self.template.format(**kwargs)
class CustomOutputParser(AgentOutputParser):
begin: bool = False
def __init__(self):

5
server/chat/agent_chat.py
View File

@ -91,8 +91,11 @@ async def agent_chat(query: str = Body(..., description="用户输入", examples
llm=model,
tools=tools,
callback_manager=None,
verbose=True,
# Langchain Prompt is not constructed directly here, it is constructed inside the GLM3 agent.
prompt=prompt_template,
input_variables=["input", "intermediate_steps", "history"],
memory=memory,
verbose=True,
)
else:
agent = LLMSingleActionAgent(

2
server/utils.py
View File

@ -9,7 +9,7 @@ from configs import (LLM_MODELS, LLM_DEVICE, EMBEDDING_DEVICE,
FSCHAT_MODEL_WORKERS, HTTPX_DEFAULT_TIMEOUT)
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from langchain.chat_models import ChatOpenAI, AzureChatOpenAI, ChatAnthropic
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI, AzureOpenAI, Anthropic
import httpx
from typing import Literal, Optional, Callable, Generator, Dict, Any, Awaitable, Union