Java:大文件拆分工具
java大文件拆分工具(过滤掉表头)
import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.FileReader; import java.io.FileWriter; import java.io.IOException; public class SplitTools { public static void main(String[] args) throws IOException { // "E:\\eutranrelation.csv"; String bigFilePath = args[0]; // "E:\\eutranrelation-%s.csv"; String splitFileParttern = args[1]; // 32 int splitFileCount = Integer.valueOf(args[2]); BufferedReader reader = new BufferedReader(new FileReader(bigFilePath)); String header = reader.readLine(); int totalLine = 0; if (header != null) { String line = null; while ((line = reader.readLine()) != null) { totalLine++; } } reader.close(); int splitFileTotalLines = totalLine / splitFileCount; int generateFileIdx = 0; BufferedWriter bufferedWriter = null; reader = new BufferedReader(new FileReader(bigFilePath)); header = reader.readLine(); int currentIdx = 0; if (header != null) { String line = null; while ((line = reader.readLine()) != null) { if (currentIdx == 0) { String filePath = String.format(splitFileParttern, String.valueOf(generateFileIdx)); bufferedWriter = new BufferedWriter( new FileWriter(filePath)); } bufferedWriter.write(line + "\r\n"); currentIdx++; if (currentIdx == splitFileTotalLines) { generateFileIdx++; if (generateFileIdx != splitFileCount) { currentIdx = 0; bufferedWriter.flush(); bufferedWriter.close(); } } } if (bufferedWriter != null) { bufferedWriter.flush(); bufferedWriter.close(); } } reader.close(); System.out.println("The total number of documents is:" + totalLine); } }
基础才是编程人员应该深入研究的问题,比如:
1)List/Set/Map内部组成原理|区别
2)mysql索引存储结构&如何调优/b-tree特点、计算复杂度及影响复杂度的因素。。。
3)JVM运行组成与原理及调优
4)Java类加载器运行原理
5)Java中GC过程原理|使用的回收算法原理
6)Redis中hash一致性实现及与hash其他区别
7)Java多线程、线程池开发、管理Lock与Synchroined区别
8)Spring IOC/AOP 原理;加载过程的。。。
【+加关注】。