llama-cpp-agent 通过结构化agent 生成数据

以前说过基于结构化数据配置进行函数调用的，以下是直接使用llama-cpp-agent 自带的StructuredOutputAgent

环境准备

需要安装llama-cpp-agent 以及启动一个llama-server

安装

pip install llama-cpp-agent

服务启动
Linux 环境自己编译的llama-server

llama-server  -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf  --host 0.0.0.0

llama-cpp-agent 代码调用

demo3.py

from llama_cpp_agent import StructuredOutputAgent

from llama_cpp_agent.providers import LlamaCppServerProvider

from  pydantic  import BaseModel
 
provider = LlamaCppServerProvider("http://localhost:8080")

# 注意我们可以通过pydantic 进一步对于类型进行说明，使用上比较类似instructor 

class User(BaseModel):

    name: str

    age: int
 
agent = StructuredOutputAgent(

    llama_llm=provider,

    debug_output=True

    )

txt = "大龙，今年30岁了"

msg = agent.create_object(User,txt)

print(msg)

效果

<|im_start|>system

Read and follow the instructions below:
 
<system_instructions>

You are an advanced AI agent. You are tasked to assist the user by creating structured output in JSON format.
 
Output Model: User

  Fields:

    name (str)

    age (int)

</system_instructions>
 
Your output should be structured as JSON and represent one of the following output models: 
 
<output_models>

Output Model: User

  Fields:

    name (str)

    age (int)

</output_models>
 
Your JSON output should have the following fields:
 
The 'model' field should contain the name of the specific model that you are outputting, based on the system instructions. 
 
The 'fields' field should contain the actual fields and content of the model you are outputting, filled out according to the system instructions.<|im_end|>

<|im_start|>user

Create an JSON response based on the following input.
 
Input:
 
大龙，今年30岁了<|im_end|>

<|im_start|>assistant

{

  "model": 

    "User",

  "fields": {

    "name": "大龙",

    "age": 30

  }

}

name='大龙' age=30