llama-cpp-agent 通过构结构化输出实现函数调用
主要是一个简单测试，记录下
环境准备
需要安装llama-cpp-agent 以及启动一个llama-server
- 安装
pip install llama-cpp-agent
- 服务启动
Linux 环境自己编译的llama-server
llama-server -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf --host 0.0.0.0
llama-cpp-agent 代码调用
- demo.py
from llama_cpp import Llama
from llama_cpp_agent import LlamaCppAgent
from llama_cpp_agent.providers import LlamaCppServerProvider
from llama_cpp_agent.llm_output_settings import LlmStructuredOutputSettings
provider = LlamaCppServerProvider("http://localhost:8080")
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示
def add(a:int,b:int) ->int:
"""
add two numbers
Args:
a: int
b: int
"""
return a+b
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示
def sub(a:int,b:int) ->int:
"""
sub two numbers
Args:
a: int
b: int
"""
return a - b
# 通过结构化输出进行函数定义
output_settings = LlmStructuredOutputSettings.from_functions([add,sub],allow_parallel_function_calling=True)
agent = LlamaCppAgent(
provider=provider,
debug_output=True,
system_prompt="你是一个强大的人工智能助手,通过基于json格式进行函数调用",
)
msg = agent.get_chat_response("200 加 300 的结果",structured_output_settings=output_settings)
print(msg)
- 输出效果
<|im_start|>system
Read and follow the instructions below:
<system_instructions>
你是一个强大的人工智能助手,通过基于json格式进行函数调用
</system_instructions>
You can call functions to help you with your tasks and user queries. The available functions are:
<function_list>
Function: add
Description: add two numbers
Parameters:
a (int): int
b (int): int
Function: sub
Description: sub two numbers
Parameters:
a (int): int
b (int): int
</function_list>
To call a function, respond with a JSON object (to call one function) or a list of JSON objects (to call multiple functions), with each object containing these fields:
- "function": Put the name of the function to call here.
- "arguments": Put the arguments to pass to the function here.
The result of each function call will be returned to you before you need to respond again.<|im_end|>
<|im_start|>user
200 加 300 的结果<|im_end|>
<|im_start|>assistant
[
{
"function":
"add",
"arguments": {
"a": 200,
"b": 300
}
}
]
[{'function': 'add', 'arguments': {'a': 200, 'b': 300}, 'return_value': 500}]
说明
基于提示词结构化输出的函数调用相比直接函数调用还是一种相对稳定的方法， 目前一些开源模型使用内置的函数调用稳定性不太好
参考资料
https://github.com/Maximilian-Winter/llama-cpp-agent
https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
https://llama-cpp-agent.readthedocs.io/en/latest/
