Kaldi如何准备自己的数据
Introduction
跑完kaldi的一些脚本例子，你可能想要自己用Kaldi跑自己的数据集。这里将会阐述如何准备好数据。
run.sh较上的部分是有关数据准备的，通常local与数据集相关。
例如：RM数据集
local/rm_data_prep.sh /export/corpora5/LDC/LDC93S3A/rm_comp || exit 1; utils/prepare_lang.sh data/local/dict '!SIL' data/local/lang data/lang || exit 1; local/rm_prepare_grammar.sh || exit 1;
再例如：再WSJ数据集
wsj0=/export/corpora5/LDC/LDC93S6B
wsj1=/export/corpora5/LDC/LDC94S13B
local/wsj_data_prep.sh $wsj0/??-{?,??}.? $wsj1/??-{?,??}.? || exit 1;
local/wsj_prepare_dict.sh || exit 1;
utils/prepare_lang.sh data/local/dict "<SPOKEN_NOISE>" data/local/lang_tmp data/lang || exit 1;
local/wsj_format_data.sh || exit 1;
WSJ相对RM多的命令与训练的语言模型有关，但是最重要的还是两者共有的命令。
数据准备阶段输出包含两个部分：一个与data有关( data/train/)，一个与language有关(data/lang/)。data与具体的录音数据有关，lang与语言本身有关
(lexicon, phone等)，如果你想要用现有的系统和语言模型解码准备好的数据，则你需要进一步熟悉它们。
Data preparation-- the "data" part.
可以看下egs/swbd/s5， data/train 目录：
s5# ls data/train cmvn.scp feats.scp reco2file_and_channel segments spk2utt text utt2spk wav.scp
所有文件都很重要，对于简单的任务，是没有seg信息的(发音直接对应于一个文件)。
"utt2spk", "text" and "wav.scp" and possibly "segments" and "reco2file_and_channel"这些需要你自己创建，剩下的可以用标准脚本生成。
如果文件命令规范的话，很多可以用脚本自动生成。
Files you need to create yourself
text文件记录了每个发音id与其对应的文本。
s5# head -3 data/train/text sw02001-A_000098-001156 HI UM YEAH I'D LIKE TO TALK ABOUT HOW YOU DRESS FOR WORK AND sw02001-A_001980-002131 UM-HUM sw02001-A_002736-002893 AND IS
发音id:语音库名称，说话人id作为前缀,语音时间戳信息000098-001156
2001-A作为前缀;这样的好处是有助与与说话人信息相关的排序(utt2spk and spk2utt)。
有时我们需要将说话人id与发音分离开，用-是非常安全的，因为它是最小的ASCII值，如果说话人Id长度变化，某些情况排序过程会终止，有可能会崩溃的(按照C排序)。
另外一个重要的文件是wav.scp。
s5# head -3 data/train/wav.scp sw02001-A /home/dpovey/kaldi-trunk/tools/sph2pipe_v2.5/sph2pipe -f wav -p -c 1 /export/corpora3/LDC/LDC97S62/swb1/sw02001.sph | sw02001-B /home/dpovey/kaldi-trunk/tools/sph2pipe_v2.5/sph2pipe -f wav -p -c 2 /export/corpora3/LDC/LDC97S62/swb1/sw02001.sph |
文件格式：
<recording-id> <extended-filename>
extended-filename有可能是真实的wav文件，如果seg文件不存在，则wav.scp每一行的第一个标识是发音id.
wav.scp必须是单声道的，如果文件是多声道的，可以用sox提取出指定的声道。
"segments" 文件：
s5# head -3 data/train/segments sw02001-A_000098-001156 sw02001-A 0.98 11.56 sw02001-A_001980-002131 sw02001-A 19.8 21.31 sw02001-A_002736-002893 sw02001-A 27.36 28.93
<utterance-id> <recording-id> <segment-begin> <segment-end>
秒为单位，
"reco2file_and_channel" 仅仅会被用到，当NIST sclite工具评分时。
s5# head -3 data/train/reco2file_and_channel sw02001-A sw02001 A sw02001-B sw02001 B sw02005-A sw02005 A
<recording-id> <filename> <recording-side (A or B)>
如果你没有stm文件，则你不需要知道其它的，也不需要用到reco2file_and_channel文件
utt2spk"文件
s5# head -3 data/train/utt2spk sw02001-A_000098-001156 2001-A sw02001-A_001980-002131 2001-A sw02001-A_002736-002893 2001-A
The format is
<utterance-id> <speaker-id>
spk2gender文件
s5# head -3 ../../rm/s5/data/train/spk2gender adg0 f ahh0 m ajp0 m
说话人id 性别
export LC_ALL=C 排序方式
Files you don't need to create yourself
"spk2utt" file
utils/utt2spk_to_spk2utt.pl data/train/utt2spk > data/train/spk2utt
feats.scp file.指出了发音id，其对应的mfcc特征位于ark文件的位置s5# head -3 data/train/feats.scp sw02001-A_000098-001156 /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/raw_mfcc_train.1.ark:24 sw02001-A_001980-002131 /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/raw_mfcc_train.1.ark:54975 sw02001-A_002736-002893 /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/raw_mfcc_train.1.ark:62762
<utterance-id> <extended-filename-of-features>
This feats.scp file是由下面的命令创建的
steps/make_mfcc.sh --nj 20 --cmd "$train_cmd" data/train exp/make_mfcc/train $mfccdir
cmvn.scp 说话人id 特征ark位置
s5# head -3 data/train/cmvn.scp 2001-A /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/cmvn_train.ark:7 2001-B /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/cmvn_train.ark:253 2005-A /home/dpovey/kaldi-trunk/egs/swbd/s5/mfcc/cmvn_train.ark:499
cmvn.scp由下面命令创建
steps/compute_cmvn_stats.sh data/train exp/make_mfcc/train $mfccdir
验证、修复 文件格式
utils/validate_data_dir.sh data/train
utils/fix_data_dir.sh data/train
Data preparation-- the "lang" directory.
lang文件夹下的内容：
s5# ls data/lang L.fst L_disambig.fst oov.int oov.txt phones phones.txt topo words.txt
拷贝lang的所有文件，加上G.fst存入lang_test中
s5# ls data/lang_test
G.fst L.fst L_disambig.fst oov.int oov.txt phones phones.txt topo words.txt
lang_test/ was created by copying lang/ and adding G.fst.
s5# ls data/lang/phones context_indep.csl disambig.txt nonsilence.txt roots.txt silence.txt context_indep.int extra_questions.int optional_silence.csl sets.int word_boundary.int context_indep.txt extra_questions.txt optional_silence.int sets.txt word_boundary.txt disambig.csl nonsilence.csl optional_silence.txt silence.csl
phones里面的文件非常多，幸运的时，作为kaldi用户，你不需要自己创建这些文件，只需要一些简单的输入，执行utils/prepare_lang.sh脚本就可完成。
Fortunately you, as a Kaldi user, don't have to create all of these files because we have a script "utils/prepare_lang.sh" that creates it all for you based on simpler inputs
语音识别过程中需要对声学模型进行构图，即扩展HCLG的过程，
扩展是按照H<-C<-L<-G的顺序进行的,
首先扩展G,
1.G.fst: The Language Model FST
FSA grammar，可以通过n-gram构建得到，即把字构成了词组
2.L_disambig.fst: The Phonetic Dictionary with Disambiguation Symbols FST
构建一个FST(LG)，输入时phone，输出是word，即把phone转化成了字The file L.fst is the Finite State Transducer form of the lexicon with phone symbols on the input and word symbols on the output.
A lexicon with disambiguation symbols
3.C.fst: The Context FST
把triphone 转化成monophone,即在第2步骤中扩展了context，即扩展triphone,最终输出是CLG
4.H.fst: The HMM FST
把HMM的state映射到triphone ,即把HMM的pdf-id映射到triphone,也就是扩展了HMM,
即输入时pdf-id，输出是word,也就是HCLG
HCLG.fst: final graph
把步骤1-4合起来HCLG，就是构图中构建WFST的过程。
即，输入是pdf-id，输出是对应的词组
Contents of the "lang" directory
phones.txt和words.txt都是OpenFst格式的符号文件，每一行内容是文本与对应的数字。
phones.txtandwords.txt. These are both symbol-table files, in the OpenFst format
each line is the text form and then the integer form:
s5# head -3 data/lang/phones.txt <eps> 0 SIL 1 SIL_B 2 s5# head -3 data/lang/words.txt <eps> 0 !SIL 1 -'S 2
这两个文件会被int2sym.pl和sym2int.pl脚本以及fstcompile和fstprint命令使用到。
They are mostly only accessed by the scripts utils/int2sym.pl and utils/sym2int.pl, and by the OpenFst programs fstcompile and fstprint.
L.fst时FST形式的lexicon，详细可参考下面的论文。
The file L.fst is the Finite State Transducer form of the lexicon。见 (L, see "Speech Recognition with Weighted Finite-State Transducers" by Mohri, Pereira and Riley, in Springer Handbook on SpeechProcessing and Speech Communication, 2008).
L_disambig.fst也是lexicon，不过包含的时歧义的符号#1,#2。
L_disambig.fst is the lexicon, as above but including the disambiguation symbols #1, #2
The file data/lang/oov.txt contains just a single line:
oov.txt仅仅包含一行。
s5# cat data/lang/oov.txt <UNK>
在训练过程中，这个词不在发音词典里。我们称之为垃圾音素，这个音素与各种不同的spoken noise对齐。
This is the word that we will map all out-of-vocabulary words to during training，we designate as a "garbage phone"; this phone will align with various kinds of spoken noise.<SPN> (short for "spoken noise"):
s5# grep -w UNK data/local/dict/lexicon.txt <UNK> SPN
oov.int对应oov.txt的整数形式。
The fileoov.intcontains the integer form of this
topo描述的时HMM的拓扑结构。
The file data/lang/topo ;This specifies the topology of the HMMs
s5# cat data/lang/topo <Topology> <TopologyEntry> <ForPhones> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 </ForPhones> <State> 0 <PdfClass> 0 <Transition> 0 0.75 <Transition> 1 0.25 </State> <State> 1 <PdfClass> 1 <Transition> 1 0.75 <Transition> 2 0.25 </State> <State> 2 <PdfClass> 2 <Transition> 2 0.75 <Transition> 3 0.25 </State> <State> 3 </State> </TopologyEntry> <TopologyEntry> <ForPhones> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 </ForPhones> <State> 0 <PdfClass> 0 <Transition> 0 0.25 <Transition> 1 0.25 <Transition> 2 0.25 <Transition> 3 0.25 </State> <State> 1 <PdfClass> 1 <Transition> 1 0.25 <Transition> 2 0.25 <Transition> 3 0.25 <Transition> 4 0.25 </State> <State> 2 <PdfClass> 2 <Transition> 1 0.25 <Transition> 2 0.25 <Transition> 3 0.25 <Transition> 4 0.25 </State> <State> 3 <PdfClass> 3 <Transition> 1 0.25 <Transition> 2 0.25 <Transition> 3 0.25 <Transition> 4 0.25 </State> <State> 4 <PdfClass> 4 <Transition> 4 0.75 <Transition> 5 0.25 </State> <State> 5 </State> </TopologyEntry> </Topology>
查看头3行
s5# head -3 data/lang/phones/context_indep.txt 上下文无关的音素 文本表示
SIL
SIL_B
SIL_E
s5# head -3 data/lang/phones/context_indep.int 整数表示
1 2 3
s5# cat data/lang/phones/context_indep.csl 所有音素，整数表示
1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20
context_indep.txt包含一些音素，我们用这些建立与上下文无关的模型，我们不会构建有关左音素和右音素的决策树，事实上，我们会构建较小的树，这些树只会问与中心音素与HMM状态相关的问题，
创建这些树依赖于root.txt
"context_indep.txt" contains a list of those phones for which we build context-independent models
we do not build a decision tree that gets to ask questions about the left and right phonetic context
In fact, we do build smaller trees where we get to ask questions about the central phone and the HMM-state，
this depends on the "roots.txt" file
root.txt包含一些信息，它与我们如何构建与音素相关的决策树。
The fileroots.txtcontains information that relates to how we build the phonetic-context decision tree:
shared表示共享中心节点，split表示分裂成以下几种节点，如图所示。
head data/lang/phones/roots.txt shared split SIL SIL_B SIL_E SIL_I SIL_S shared split SPN SPN_B SPN_E SPN_I SPN_S shared split NSN NSN_B NSN_E NSN_I NSN_S shared split LAU LAU_B LAU_E LAU_I LAU_S ... shared split B_B B_E B_I B_S
In addition, all three states of a HMM (or all five states, for silences) share the root
# cat data/lang/phones/context_indep.txt SIL SIL_B SIL_E SIL_I SIL_S SPN SPN_B SPN_E SPN_I SPN_S NSN NSN_B NSN_E NSN_I NSN_S LAU LAU_B LAU_E LAU_I LAU_S
这些音素的变体与词位置有关，并不是所有的变体会被用到。
There are a lot of variants of these phones because of word-position dependency;
not all of these variants will ever be used
eg: SIL 变体SIL_BSIL_ISIL_ESIL_S
我们区别开silence.txt和nonsilence.txt，silence.txt的音素都会被用于各种线性变换，如LDA,MLLT,fMLLR，而nonsilence不会的。
What we mean by "nonsilence" phones is, phones that we intend to estimate various kinds of linear transforms on: that is, global transforms such as LDA and MLLT, and speaker adaptation transforms such as fMLLR
does not pay to include silence in the estimation of such transforms
s5# head -3 data/lang/phones/silence.txt SIL SIL_B SIL_E s5# head -3 data/lang/phones/nonsilence.txt IY_B IY_E IY_I
s5# head -3 data/lang/phones/disambig.txt #0 #1 #2
optional_silence.txt只包含一个音素，这个音素有选择的位于两个词之间。
The fileoptional_silence.txtcontains a single phone which can optionally appear between word
s5# cat data/lang/phones/optional_silence.txt SIL
set.txt 的每一行，组合排列着某个音素的所有变体。
sets.txtgroups together all the word-position-dependent versions of each phone
s5# head -3 data/lang/phones/sets.txt SIL SIL_B SIL_E SIL_I SIL_S SPN SPN_B SPN_E SPN_I SPN_S NSN NSN_B NSN_E NSN_I NSN_S
extra_questions.txt问题集，人工的，用于构建决策树。
s5# cat data/lang/phones/extra_questions.txt
IY_B B_B D_B F_B G_B K_B SH_B L_B M_B N_B OW_B AA_B TH_B P_B OY_B R_B UH_B AE_B S_B T_B AH_B V_B W_B Y_B Z_B CH_B AO_B DH_B UW_B ZH_B EH_B AW_B AX_B EL_B AY_B EN_B HH_B ER_B IH_B JH_B EY_B NG_B
IY_E B_E D_E F_E G_E K_E SH_E L_E M_E N_E OW_E AA_E TH_E P_E OY_E R_E UH_E AE_E S_E T_E AH_E V_E W_E Y_E Z_E CH_E AO_E DH_E UW_E ZH_E EH_E AW_E AX_E EL_E AY_E EN_E HH_E ER_E IH_E JH_E EY_E NG_E
IY_I B_I D_I F_I G_I K_I SH_I L_I M_I N_I OW_I AA_I TH_I P_I OY_I R_I UH_I AE_I S_I T_I AH_I V_I W_I Y_I Z_I CH_I AO_I DH_I UW_I ZH_I EH_I AW_I AX_I EL_I AY_I EN_I HH_I ER_I IH_I JH_I EY_I NG_I
IY_S B_S D_S F_S G_S K_S SH_S L_S M_S N_S OW_S AA_S TH_S P_S OY_S R_S UH_S AE_S S_S T_S AH_S V_S W_S Y_S Z_S CH_S AO_S DH_S UW_S ZH_S EH_S AW_S AX_S EL_S AY_S EN_S HH_S ER_S IH_S JH_S EY_S NG_S
SIL SPN NSN LAU
SIL_B SPN_B NSN_B LAU_B
SIL_E SPN_E NSN_E LAU_E
SIL_I SPN_I NSN_I LAU_I
SIL_S SPN_S NSN_S LAU_S
The first four questions are asking about the word-position,
last five do the same for the "silence phones"
word_boundary.txt 解释这些音素与词位置的关系。
The file word_boundary.txt explains how the phones relate to word positions:
s5# head data/lang/phones/word_boundary.txt SIL nonword SIL_B begin SIL_E end SIL_I internal SIL_S singleton SPN nonword SPN_B begin
Creating the language model or grammar
文档出处：http://www.kaldi-asr.org/doc/data_prep.html
