1 参考文献

【官方指引】https://qwen.readthedocs.io/en/latest/

【ModelScope训练】https://modelscope.cn/docs/%E4%BD%BF%E7%94%A8Tuners

【CUDA下载安装教程】https://blog.csdn.net/changyana/article/details/135876568

【安装cuDNN】https://developer.nvidia.com/rdp/cudnn-archive

【安装PyTorch】https://pytorch.org/

【安装Ollama】https://ollama.com/download

2 基础环境

2.1 安装CUDA

【查看显卡驱动】nvidia-smi

【验证CUDA安装】nvcc -V

首先查看NVIDIA显卡对应的CUDA版本号，然后根据此版本号下载对应版本的Toolkit

2.2 安装cuDNN

把下载的cudnn压缩包进行解压，在cudnn的文件夹下，把bin，include，lib文件夹下的内容对应拷贝到cuda相应的bin，include，lib下即可，最后安装完成。

2.3 安装PyTorch

# 验证PyTorch是否与CUDA兼容

import torch

print(torch.__version__)
print(torch.cuda.is_available())

3 通过Ollama调用模型

Ollama是一种比较简单方便的本地运行方案。通过Ollama官网直接安装，可搜索支持的大模型：https://ollama.com/search?q=qwen

4 Qwen官方指引

Qwen官方指南详细说明了支持的各种方案，比查看各种网络博客说明更清晰。

5 自定义训练

5.1 参考ModelScope指引

# 自定义ModelScope模型缓存路径
export MODELSCOPE_CACHE=/Users/kuliuheng/workspace/aiWorkspace/Qwen

 1 # A100 18G memory
 2 from swift import Seq2SeqTrainer, Seq2SeqTrainingArguments
 3 from modelscope import MsDataset, AutoTokenizer
 4 from modelscope import AutoModelForCausalLM
 5 from swift import Swift, LoraConfig
 6 from swift.llm import get_template, TemplateType
 7 import torch
 8 
 9 pretrained_model = 'qwen/Qwen2.5-0.5B-Instruct'
10 
11 
12 def encode(example):
13     inst, inp, output = example['instruction'], example.get('input', None), example['output']
14     if output is None:
15         return {}
16     if inp is None or len(inp) == 0:
17         q = inst
18     else:
19         q = f'{inst}\n{inp}'
20     example, kwargs = template.encode({'query': q, 'response': output})
21     return example
22 
23 
24 if __name__ == '__main__':
25     # 拉起模型
26     model = AutoModelForCausalLM.from_pretrained(pretrained_model, torch_dtype=torch.bfloat16, device_map='auto', trust_remote_code=True)
27     lora_config = LoraConfig(
28                     r=8,
29                     bias='none',
30                     task_type="CAUSAL_LM",
31                     target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
32                     lora_alpha=32,
33                     lora_dropout=0.05)
34     model = Swift.prepare_model(model, lora_config)
35     tokenizer = AutoTokenizer.from_pretrained(pretrained_model, trust_remote_code=True)
36     dataset = MsDataset.load('AI-ModelScope/alpaca-gpt4-data-en', split='train')
37     template = get_template(TemplateType.chatglm3, tokenizer, max_length=1024)
38 
39     dataset = dataset.map(encode).filter(lambda e: e.get('input_ids'))
40     dataset = dataset.train_test_split(test_size=0.001)
41 
42     train_dataset, val_dataset = dataset['train'], dataset['test']
43 
44     train_args = Seq2SeqTrainingArguments(
45         output_dir='output',
46         learning_rate=1e-4,
47         num_train_epochs=2,
48         eval_steps=500,
49         save_steps=500,
50         evaluation_strategy='steps',
51         save_strategy='steps',
52         dataloader_num_workers=4,
53         per_device_train_batch_size=1,
54         gradient_accumulation_steps=16,
55         logging_steps=10,
56     )
57 
58     trainer = Seq2SeqTrainer(
59         model=model,
60         args=train_args,
61         data_collator=template.data_collator,
62         train_dataset=train_dataset,
63         eval_dataset=val_dataset,
64         tokenizer=tokenizer)
65 
66     trainer.train()

View Code

（1）官方示例代码中没有写 __main__ 主函数入口，实际运行时发现会报错提示说：子线程在主线程尚未完成初始化之前就运行了。所以这里就补齐了一个主函数入口

（2）官方代码中没有针对 'qwen/Qwen2.5-0.5B-Instruct' 模型代码，运行时target_modules会提示错误，需要指定模型中实际存在的模块名才行。这里有个技巧，通过打印模型信息可以看到实际的层级结构：

from modelscope import AutoModelForCausalLM

model_name = 'qwen/Qwen2.5-0.5B-Instruct'
model = AutoModelForCausalLM.from_pretrained(model_name)
print(model)

得到如下结果：

Qwen2ForCausalLM(
  (model): Qwen2Model(
    (embed_tokens): Embedding(151936, 896)
    (layers): ModuleList(
      (0-23): 24 x Qwen2DecoderLayer(
        (self_attn): Qwen2SdpaAttention(
          (q_proj): Linear(in_features=896, out_features=896, bias=True)
          (k_proj): Linear(in_features=896, out_features=128, bias=True)
          (v_proj): Linear(in_features=896, out_features=128, bias=True)
          (o_proj): Linear(in_features=896, out_features=896, bias=False)
          (rotary_emb): Qwen2RotaryEmbedding()
        )
        (mlp): Qwen2MLP(
          (gate_proj): Linear(in_features=896, out_features=4864, bias=False)
          (up_proj): Linear(in_features=896, out_features=4864, bias=False)
          (down_proj): Linear(in_features=4864, out_features=896, bias=False)
          (act_fn): SiLU()
        )
        (input_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
        (post_attention_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
      )
    )
    (norm): Qwen2RMSNorm((896,), eps=1e-06)
    (rotary_emb): Qwen2RotaryEmbedding()
  )
  (lm_head): Linear(in_features=896, out_features=151936, bias=False)
)