初步体验 llama.cpp

llama.cpp: Port of Facebook's LLaMA model in C/C++

github 仓库：https://github.com/ggerganov/llama.cpp

参考博文：High-Speed Inference with llama.cpp and Vicuna on CPU

第1步，准备一台阿里云4核8G的服务器，操作系统用的是 ubuntu 22.04

第2步，签出 llama.cpp 源码进行 build

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

build 过程中开始部分的输出

I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_K_QUANTS
I LDFLAGS:  
I CC:       cc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
I CXX:      g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0

第3步，下载 4-bit 版本的 Vicuna-7B 模型，文件大小是3.6G

cd ./models/
wget https://huggingface.co/TheBloke/vicuna-7B-1.1-GGML/resolve/main/vicuna-7b-1.1.ggmlv3.q4_0.bin

第4步，使用这个模型输入提示词 Tell me about cnblogs

./main -m ./models/vicuna-7b-1.1.ggmlv3.q4_0.bin -p "Tell me about cnblogs" -n 1024

得到的回答（只列出了第1段）：

Tell me about cnblogs.com
cnblogs.com is a hosting and blogging platform that enables users to create and maintain their own blogs with ease. The website provides a user-friendly interface for creating and managing weblogs, and offers a range of features such as custom domains, email integration, and the ability to host multiple blogs from a single account. Additionally, cnblogs.com offers security features, such as SSL encryption, to protect users' blogs and data.
....

注：从回答内容看，多数内容是编造出来的。

第5步，下载 8-bit 版本的 Vicuna-7B 模型，文件大小是6.7G

cd ./models/
wget https://huggingface.co/TheBloke/vicuna-7B-1.1-GGML/resolve/main/vicuna-7b-1.1.ggmlv3.q8_0.bin

第6步，用同样的提示词提问

./main -m ./models/vicuna-7b-1.1.ggmlv3.q8_0.bin -p "Tell me about cnblogs" -n 1024

回答的内容是更离谱的编造：

Tell me about cnblogs:
cnblogs is the perfect platform for companies and organizations to share news, updates, and information with their followers. Whether you want to highlight employee achievements, announce new products or services, or provide valuable insights into your industry, our blogging service can help you get your message out there. Our team of experienced writers can create compelling content that is optimized for search engines and tailored to your specific needs. With cnblogs, you can build a loyal following and establish yourself as a thought leader in your field.
What are the benefits of using cnblogs? [end of text]

第6步，改为使用交互模式

./main -m ./models/vicuna-7b-1.1.ggmlv3.q4_0.bin -p "Tell me about cnblogs" -n 256 --repeat_penalty 1.0 --color -i -r "User:"

对于默认提示词的回答变成了：

Tell me about cnblogs.com
cnblogs.com is a domain name that has been around since at least 2004. There is no information available about who owns or operates the site. It is possible that it is a personal blog or a small website that has not been actively maintained in some time.

在交互模式下输入提示词 Tell me about github，回答内容好多了：

Tell me about github

GitHub is a web-based hosting service for version control using Git. It was founded in 2008 and was acquired by Microsoft in 2018. It provides a platform for developers to share code and collaborate on projects, and it also allows for code reviews and has a large community of developers. GitHub is a popular platform for open-source projects, and it is used by many large companies and organizations. It is also widely used by individual developers and small teams.

初步体验到此结束。