llama-cpp-agent 通过构结构化输出实现函数调用

主要是一个简单测试，记录下

环境准备

需要安装llama-cpp-agent 以及启动一个llama-server

安装

pip install llama-cpp-agent

服务启动
Linux 环境自己编译的llama-server

llama-server  -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf  --host 0.0.0.0

llama-cpp-agent 代码调用

demo.py

from llama_cpp import Llama

from llama_cpp_agent import LlamaCppAgent

from llama_cpp_agent.providers import LlamaCppServerProvider
 
from  llama_cpp_agent.llm_output_settings import LlmStructuredOutputSettings
 
provider = LlamaCppServerProvider("http://localhost:8080")
 
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示
 
def add(a:int,b:int) ->int:

    """

    add two numbers

    Args:

        a: int

        b: int

    """

    return a+b
 
# 注意函数注释以及类型定义是必须的，否则运行会有错误提示

def sub(a:int,b:int) ->int:

    """

    sub two numbers

    Args:

        a: int

        b: int

    """

    return a - b

# 通过结构化输出进行函数定义

output_settings = LlmStructuredOutputSettings.from_functions([add,sub],allow_parallel_function_calling=True)
 
agent = LlamaCppAgent(

    provider=provider,

    debug_output=True,

    system_prompt="你是一个强大的人工智能助手,通过基于json格式进行函数调用",

    )
 
msg = agent.get_chat_response("200 加 300 的结果",structured_output_settings=output_settings)

print(msg)

输出效果

<|im_start|>system

Read and follow the instructions below:
 
<system_instructions>

你是一个强大的人工智能助手,通过基于json格式进行函数调用

</system_instructions>
 
You can call functions to help you with your tasks and user queries. The available functions are:
 
<function_list>

Function: add

  Description: add two numbers

  Parameters:

    a (int): int

    b (int): int
 
Function: sub

  Description: sub two numbers

  Parameters:

    a (int): int

    b (int): int

</function_list>
 
To call a function, respond with a JSON object (to call one function) or a list of JSON objects (to call multiple functions), with each object containing these fields:
 
- "function": Put the name of the function to call here. 

- "arguments": Put the arguments to pass to the function here.
 
The result of each function call will be returned to you before you need to respond again.<|im_end|>

<|im_start|>user

200 加 300 的结果<|im_end|>

<|im_start|>assistant

[

    {

        "function": 

            "add",

        "arguments": {

            "a": 200,

            "b": 300

        }

    }

]

[{'function': 'add', 'arguments': {'a': 200, 'b': 300}, 'return_value': 500}]