使用kaggle白嫖GPU搭建chatglm-6B Qwen-7B等大模型

1、kaggle设置

1、登录kaggle

2、手机验证

不用手机验证没办法使用gpu资源

3、更换运行模式

4、网络打开

2、ChatGLM-6B环境搭建

!git clone https://github.com/THUDM/ChatGLM-6B.git

cd ChatGLM-6B

!pip install -r requirements.txt

3、运行chatglm-6b

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)

回答

你好👋！我是人工智能助手 ChatGLM-6B，很高兴见到你，欢迎问我任何问题。

4、Qwen-7B

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

# Now you do not need to add "trust_remote_code=True"
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2-7B-Instruct",
    torch_dtype="auto",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B-Instruct")

# Instead of using model.chat(), we directly use model.generate()
# But you need to use tokenizer.apply_chat_template() to format your inputs as shown below
messages = [
    {"role": "user", "content": "你是谁呢?"},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)
generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

回答

我是阿里云开发的一款超大规模语言模型，我叫通义千问。