python训练阿里云语音合成模型
训练阿里云多人训练语音合成模型
- model_link:https://modelscope.cn/models/speech_tts/speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_24k/summary
- 1.获取KANTTS算法训练框架
- 2.配置虚拟环境(conda)
需要下载pytorch及其他kantts算法依赖
- 3.数据准备:https://modelscope.cn/datasets/speech_tts/AISHELL-3/summary
- 4.数据重新采样，例如手中有48k采样率的音频文件，你需要训练的24k的模型，那么就需要把相关文件采样率转换为24k(根据具体需求完成)
转换采样率
import os
from pydub import AudioSegment
input_folder = "/path/aida/wav_24k"
output_folder = "/path/aida/new_wav_24k"
if not os.path.exists(output_folder):
os.makedirs(output_folder)
for file in os.listdir(input_folder):
if file.endswith(".wav"):
input_file = os.path.join(input_folder, file)
output_file = os.path.join(output_folder, file)
audio = AudioSegment.from_file(input_file, format="wav")
audio = audio.set_frame_rate(24000)
audio.export(output_file, format="wav")
print("转换完成！")
# 检验采样率
import wave
def get_sample_rate(file_path):
with wave.open(file_path, 'rb') as wav_file:
sample_rate = wav_file.getframerate()
return sample_rate
file_path = '/path/aida/wav_24k/002192.wav'
sample_rate = get_sample_rate(file_path)
print(f'采样率为：{sample_rate} Hz')
- 5.特征提取
# 特征提取
python kantts/preprocess/data_process.py --voice_input_dir aishell3_16k/SSB0009 --voice_output_dir training_stage/SSB0009_feats --audio_config kantts/configs/audio_config_16k.yaml --speaker SSB0009
这是特征提取后的结果
主要是注意am_train.lst/am_vilid.lst/raw_metafile.txt里面是否有内容
提取完成后需要训练am
命令提示linux:
查看显卡nvdia-msi
- 6.训练am(声学模型)
CUDA_VISIBLE_DEVICES=0 python kantts/bin/train_sambert.py --model_config speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_16k/basemodel_16k/sambert/config.yaml --resume_path speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_16k/basemodel_16k/sambert/ckpt/checkpoint_980000.pth --root_dir training_stage/SSB0009_feats --stage_dir training_stage/SSB0009_sambert_ckpt
可以添加nohup后台挂起，这个时间稍微有一点久
- 7.训练voc
CUDA_VISIBLE_DEVICES=7 python kantts/bin/train_hifigan.py --model_config /home/syl01840793/speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_24k/basemodel_24k/hifigan/config.yaml --resume_path /home/syl01840793/speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_24k/basemodel_24k/hifigan/ckpt/checkpoint_2000000.pth --root_dir /home/syl01840793/train_config_test/Training_stage/aida/feat --stage_dir /home/syl01840793/train_config_test/Training_stage/aida/hifigan
mode_config 在基础base_model的chekpoint上增加100k,10ksave
- 8.合成语音
voc结束后会输出这些东西
找到hifigan/ckpt下，最新(最大的)check_point
运行合成语音
CUDA_VISIBLE_DEVICES=0 python kantts/bin/text_to_wav.py --txt test.txt --output_dir res/SSB0009_syn --res_zip speech_sambert-hifigan_tts_zh-cn_multisp_pretrain_16k/resource.zip --am_ckpt training_stage/SSB0009_sambert_ckpt/ckpt/checkpoint_1100000.pth --voc_ckpt training_stage/SSB0009_hifigan_ckpt/ckpt/checkpoint_2100000.pth --speaker SSB0009
合成语音且通过audition来检测质量即可
