ollama llama3.1 8b openbuddy中文模型

openbuddy 对于 llama3.1 8b 模型进行了少量的中文训练实现了不错的中文理解处理，以下是使用社区提供的gguf 格式，制作一个ollama 新模型

模型制作

  • 下载模型
    下载gguf 文件，推荐使用加速工具
  • Modelfile
    参考了llama3.1 的
FROM /home/models/openbuddy-llama3.1-8b-v22.1-131k-q4_k_m.gguf
 
TEMPLATE """{{ if .Messages }}
{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
{{- if .System }}
 
{{ .System }}
{{- end }}
{{- if .Tools }}
 
You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the orginal use question.
{{- end }}<|eot_id|>
{{- end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}
 
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
 
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
 
{{ $.Tools }}
{{- end }}
 
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
 
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}
 
{{ .Content }}{{ if not $last }}<|eot_id|>{{ end }}
{{- end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
 
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}
{{- end }}
{{- end }}
{{- else }}
{{- if .System }}<|start_header_id|>system<|end_header_id|>
 
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
 
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
 
{{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
  • 构建
ollama create openbuddy-llama3.1 -f Modelfile

使用

测试的函数调用

  • appv3.py
import openai
import json
 
 
def add(a, b):
    return a + b
 
 
def sub(a, b):
    return a - b
 
 
def send_email(userid, mailContent):
    return f"send email to {userid} with content: {mailContent}"
 
 
 
def add_role(userid, roleid):
    return f"add role {roleid} to user {userid}"
 
openai.api_key = "demo"
openai.base_url = "http://localhost:11434/v1/"
 
funcs = {
    "add": add,
    "sub": sub,
    "send_email":send_email,
    "add_role":add_role
}
 
tools = [{
    "type": "function",
            "function": {
                "name": "add",
                "description": "Add two numbers together",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": {"type": "number", "description": "First number"},
                        "b": {"type": "number", "description": "Second number"}
                    },
                    "required": ["a", "b"]
                }
            }
},
    {
    "type": "function",
            "function": {
                "name": "sub",
                "description": "计算两个数的差",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": {"type": "number", "description": "First number"},
                        "b": {"type": "number", "description": "Second number"}
                    },
                    "required": ["a", "b"]
                }
            }
},
 {
    "type": "function",
            "function": {
                "name": "add_role",
                "description": "给用户添加权限",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "userid": {"type": "string", "description": "用户id"},
                        "roleid": {"type": "string", "description": "角色id"}
                    },
                    "required": ["userid", "roleid"]
                }
            }
},
 {
    "type": "function",
            "function": {
                "name": "send_email",
                "description": "给用户发送邮件",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "userid": {"type": "string", "description": "用户id"},
                        "mailContent": {"type": "string", "description": "邮件内容"}
                    },
                    "required": ["userid", "mailContent"]
                }
            }
}
]
response = openai.chat.completions.create(
    stream=False,
    model="rongfengliang/openbuddy-llama3.1",
    messages=[
        {"role": "system", "content": "你是一个人工智能助手，你可以帮助用户处理数据，发送邮件等，但是你必须使用中文"},
        {"role": "user", "content": "给用户001发送数据处理已经完成的邮件通知同时添加角色0002"},
    ],
    tools=tools,
    tool_choice="auto",
)
message = response.choices[0].message
print(message)
if message.tool_calls:
    result = response.choices[0].message.tool_calls[0].function
    print(result)
    func = funcs.get(result.name)
    params = json.loads(result.arguments)
    print(func(**params))
else:
    print(message.content)
 
  • 效果

说明

目前基于openbuddy支持中文的进行函数调用比原生的是好一些，原生的有时有出现莫名其妙的中文信息

参考资料

https://openbuddy.ai/
https://github.com/OpenBuddy/OpenBuddy
https://huggingface.co/OpenBuddy/openbuddy-llama3.1-8b-v22.1-131k
https://huggingface.co/sunnyyy/openbuddy-llama3.1-8b-v22.1-131k-Q4_K_M-GGUF

posted on 2024-07-27 14:48  荣锋亮  阅读(158)  评论(0)  编辑  收藏  举报

导航