Linux大文件拆分、合并、校验

Linux大文件拆分、合并、校验

大文件在实际应用传输过程中往往经历拆分、合并的过程。文件在多次传输后有可能会损坏，在使用前可以进行完整性校验。

1.文件生成MD5码

md5sum rhel-server-6.3-x86_64-dvd.iso
d717af33dd258945e6304f9955487017  rhel-server-6.3-x86_64-dvd.iso

2.文件拆分

split -b 600m -d rhel-server-6.3-x86_64-dvd.iso redhat6.3_x

参数说明: -b 600m 表示文件以600mb为单位分割
         -d  分割后子文件名以数字作为后缀，默认两位
         redhat6.3_x 子文件前缀

3.拆分后生成子文件MD5码

for var in $(ls redhat*); do md5sum $var; done
b9440b925d9e9bc640fd23ff00e15450  redhat6.3_x00
af4985a9aa1bc557b5a05de3cdad026b  redhat6.3_x01
2433ab29a85789f4aca21421307ed788  redhat6.3_x02
8cbccc256b1da1014f07e9142cf2fc4c  redhat6.3_x03
962e79bceacef5c1fa9336e4a21ce995  redhat6.3_x04
22a68fb82aaa7c8d516897740ebf052f  redhat6.3_x05
#可以直接将MD5值重定向到文件
for var in $(ls redhat*); do md5sum $var >> s_sub.md5; done

4.子文件远程传输

scp redhat6.3_x00 redhat6.3_x01 redhat6.3_x02 redhat6.3_x03 redhat6.3_x04 redhat6.3_x05 user@192.168.122.36:~

5.在接收端校验子文件完整性

#5.1 在接收端生成子文件
for var in $(ls redhat6.3_x*);do md5sum $var ;done
b9440b925d9e9bc640fd23ff00e15450  redhat6.3_x00
af4985a9aa1bc557b5a05de3cdad026b  redhat6.3_x01
2433ab29a85789f4aca21421307ed788  redhat6.3_x02
8cbccc256b1da1014f07e9142cf2fc4c  redhat6.3_x03
962e79bceacef5c1fa9336e4a21ce995  redhat6.3_x04
22a68fb82aaa7c8d516897740ebf052f  redhat6.3_x05
for var in $(ls redhat*); do md5sum $var >> d_sub.md5; done
#5.2 比较源子文件MD5码，相同说明完整
diff s_sub.md5 d_sub.md5 > /dev/null && echo true || echo false
true

6.大文件合并

cat redhat6.3_x00 redhat6.3_x01 redhat6.3_x02 redhat6.3_x03 redhat6.3_x04 redhat6.3_x05 > rhel-server-6.3-x86_64-dvd.iso

7.合并后的大文件生成MD5码与源MD5值比较，值相同表示完整无损

md5sum rhel-server-6.3-x86_64-dvd.iso
d717af33dd258945e6304f9955487017  rhel-server-6.3-x86_64-dvd.iso
# 比较略(字符串比较