llama-cpp-agent 通过结构化agent 生成数据

以前说过基于结构化数据配置进行函数调用的，以下是直接使用llama-cpp-agent 自带的StructuredOutputAgent

环境准备

需要安装llama-cpp-agent 以及启动一个llama-server

• 安装

pip install llama-cpp-agent

• 服务启动
  Linux 环境自己编译的llama-server

llama-server  -m rubra-mistral-7b-instruct-v0.3.Q4_K_M.gguf  --host 0.0.0.0

llama-cpp-agent 代码调用

• demo3.py

from llama_cpp_agent import StructuredOutputAgent

from llama_cpp_agent.providers import LlamaCppServerProvider

from  pydantic  import BaseModel

 

provider = LlamaCppServerProvider("http://localhost:8080")

# 注意我们可以通过pydantic 进一步对于类型进行说明，使用上比较类似instructor 

class User(BaseModel):

    name: str

    age: int

 

agent = StructuredOutputAgent(

    llama_llm=provider,

    debug_output=True

    )

txt = "大龙，今年30岁了"

msg = agent.create_object(User,txt)

print(msg)

• 效果

<|im_start|>system

Read and follow the instructions below:

 

<system_instructions>

You are an advanced AI agent. You are tasked to assist the user by creating structured output in JSON format.

 

Output Model: User

  Fields:

    name (str)

    age (int)

</system_instructions>

 

 

Your output should be structured as JSON and represent one of the following output models: 

 

<output_models>

Output Model: User

  Fields:

    name (str)

    age (int)

</output_models>

 

Your JSON output should have the following fields:

 

 

The 'model' field should contain the name of the specific model that you are outputting, based on the system instructions. 

 

The 'fields' field should contain the actual fields and content of the model you are outputting, filled out according to the system instructions.<|im_end|>

<|im_start|>user

Create an JSON response based on the following input.

 

Input:

 

大龙，今年30岁了<|im_end|>

<|im_start|>assistant

{

  "model": 

    "User",

  "fields": {

    "name": "大龙",

    "age": 30

  }

}

name='大龙' age=30

说明

llama-cpp-agent 基于grammars 以及json schema 的结构化数据还是比较稳定的，项目中值得试用下

参考资料

https://llama-cpp-agent.readthedocs.io/en/latest/get-started/#create-agent