llama-cpp-agent 函数调用agent 进行函数调用

与结构化调用类似，就是一个记录

环境准备

需要安装llama-cpp-agent 以及启动一个llama-server

  • 安装
pip install llama-cpp-agent
  • 服务启动
    Linux 环境自己编译的llama-server
llama-server  -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf  --host 0.0.0.0

llama-cpp-agent 代码调用

  • demo2.py
from llama_cpp_agent import FunctionCallingAgent,LlamaCppFunctionTool
from llama_cpp_agent.providers import LlamaCppServerProvider
 
from  llama_cpp_agent.llm_output_settings import LlmStructuredOutputSettings
 
provider = LlamaCppServerProvider("http://localhost:8080")
 
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示
def add(a:int,b:int) ->int:
    """
    add two numbers
    Args:
        a: int
        b: int
    """
    return a+b
 
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示
def sub(a:int,b:int) ->int:
    """
    sub two numbers
    Args:
        a: int
        b: int
    """
    return a - b
 
output_settings = LlmStructuredOutputSettings.from_functions([add,sub],allow_parallel_function_calling=True)
 
add_function_tool = LlamaCppFunctionTool(add)
sub_function_tool = LlamaCppFunctionTool(sub)
 
agent = FunctionCallingAgent(
    llama_llm=provider,
    debug_output=True,
    llama_cpp_function_tools=[add_function_tool,sub_function_tool],
    allow_parallel_function_calling=True,
    system_prompt="你是一个强大的人工智能助手,通过基于json格式进行函数调用",
    )
 
msg = agent.generate_response("我现在有500元，我买东西花了300元，我现在还有多少，你能帮我计算一下吗?")
print(msg)
  • 输出结果
<|im_start|>system
Read and follow the instructions below:
 
<system_instructions>
你是一个强大的人工智能助手,通过基于json格式进行函数调用
</system_instructions>
 
 
You can call functions to help you with your tasks and user queries. The available functions are:
 
<function_list>
Function: add
  Description: add two numbers
  Parameters:
    a (int): int
    b (int): int
 
Function: sub
  Description: sub two numbers
  Parameters:
    a (int): int
    b (int): int
 
Function: send_message
  Description: Sends a message to the user.
  Parameters:
    content (str): Content of the message to be sent.
</function_list>
 
To call a function, respond with a JSON object (to call one function) or a list of JSON objects (to call multiple functions), with each object containing these fields:
 
- "thoughts_and_reasoning": Write your thoughts and reasoning for calling the function here. Think step-by-step about what information you need.
- "function": Put the name of the function to call here. 
- "arguments": Put the arguments to pass to the function here.
 
The result of each function call will be returned to you before you need to respond again.<|im_end|>
<|im_start|>user
我现在有500元，我买东西花了300元，我现在还有多少，你能帮我计算一下吗?<|im_end|>
<|im_start|>assistant
[
  {
    "thoughts_and_reasoning": "I need to subtract the amount spent from the initial amount to find the remaining balance.",
    "function": "sub",
    "arguments": {
      "a": 500,
      "b": 300
    }
  }
]
<|im_start|>system
Read and follow the instructions below:
 
<system_instructions>
你是一个强大的人工智能助手,通过基于json格式进行函数调用
</system_instructions>
 
 
You can call functions to help you with your tasks and user queries. The available functions are:
 
<function_list>
Function: add
  Description: add two numbers
  Parameters:
    a (int): int
    b (int): int
 
Function: sub
  Description: sub two numbers
  Parameters:
    a (int): int
    b (int): int
 
Function: send_message
  Description: Sends a message to the user.
  Parameters:
    content (str): Content of the message to be sent.
</function_list>
 
To call a function, respond with a JSON object (to call one function) or a list of JSON objects (to call multiple functions), with each object containing these fields:
 
- "thoughts_and_reasoning": Write your thoughts and reasoning for calling the function here. Think step-by-step about what information you need.
- "function": Put the name of the function to call here. 
- "arguments": Put the arguments to pass to the function here.
 
The result of each function call will be returned to you before you need to respond again.<|im_end|>
<|im_start|>user
我现在有500元，我买东西花了300元，我现在还有多少，你能帮我计算一下吗?<|im_end|>
<|im_start|>assistant
[
  {
    "thoughts_and_reasoning": "I need to subtract the amount spent from the initial amount to find the remaining balance.",
    "function": "sub",
    "arguments": {
      "a": 500,
      "b": 300
    }
  }
]<|im_end|>
<|im_start|>function
Function Calling Results:
 
1. Function: "sub"
Return Value: 200<|im_end|>
<|im_start|>assistant
[
  {
    "thoughts_and_reasoning": "After subtracting the spent amount from the initial amount, you are left with 200元.",
    "function": "send_message",
    "arguments": {
      "content": "您还有200元。"
    }
  }
]
您还有200元。
[{'function': 'send_message', 'arguments': {'content': '您还有200元。'}, 'return_value': 'Message sent.'}]

说明

从实际调用的日志来看，llama-cpp-agent 为了确保函数调用的稳定性，实际上使用的还是结构化输出，并不是类似openai 的json schema 定义，只是进行了一些改进，与结构化输出的处理基本类似（还包含了一个callback）

参考资料

https://github.com/Maximilian-Winter/llama-cpp-agent
https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
https://llama-cpp-agent.readthedocs.io/en/latest/
https://llama-cpp-agent.readthedocs.io/en/latest/function-calling-agent/

posted on 2024-08-28 07:37  荣锋亮  阅读(1)  评论(0)  编辑  收藏  举报

导航