split拆分大文件

最近有一个比较大的文件总行数3.5亿行,数据内容要规律,需要导入到mysql中

[t1@test01 tmp]# wc -l  winter1224.txt 
356336714 winter1224.txt

 如果直接导入,有搞垮数据库的风险,现通过split将其拆分为10个不同的文件

具体命令

split -l 30000000 -d winter1224.txt winter1224_

  

说明: -l 代表行数

           -d 代表以数字做为后缀

 

执行后结果如下:

[t1@test01 tmp]$ ls
winter1224_01  winter1224_03  winter1224_05  winter1224_07  winter1224_09  winter1224_11
winter1224_00     winter1224_02  winter1224_04  winter1224_06  winter1224_08  winter1224_10  winter1224.txt

  

验证结果:

[t1@test01 tmp]$ wc -l winter1224_*
   30000000 winter1224_00
   30000000 winter1224_01
   30000000 winter1224_02
   30000000 winter1224_03
   30000000 winter1224_04
   30000000 winter1224_05
   30000000 winter1224_06
   30000000 winter1224_07
   30000000 winter1224_08
   30000000 winter1224_09
   30000000 winter1224_10
   26336714 winter1224_11
  356336714 total
[t1@test01 tmp]$ wc -l winter1224.txt
356336714 winter1224.txt

 文件行数一致。完成ok。

 

posted @ 2021-12-26 22:28  MR__Wang  阅读(173)  评论(0编辑  收藏  举报