C++大数据的读写
当一个文件1G以上的这种,使用内存文件映射会提高读写效率;
下边时段出自《windows核心编程》,读取一个大文件,然后统计里边字符出现次数的函数:
__int64 CountOs(void) { // Get system granularity SYSTEM_INFO sinf; GetSystemInfo(&sinf); // open the data file HANDLE hFile = CreateFile(TEXT("C:\\1.TXT"), GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL); // create the file-mapping object. HANDLE hFileMapping = CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, NULL); DWORD dwFileSizeHight; __int64 qwFileSize = GetFileSize(hFile, &dwFileSizeHight); qwFileSize += (((__int64)dwFileSizeHight) << 32); // we no longer need access to the file object's handle. CloseHandle(hFile); __int64 qwFileOffset = 0, qwNumOf0s = 0; while (qwFileSize > 0) { DWORD dwBytesInBlock = sinf.dwAllocationGranularity; if (qwFileSize < sinf.dwAllocationGranularity) dwBytesInBlock = (DWORD)qwFileSize; PBYTE pbFile = (PBYTE)MapViewOfFile(hFileMapping, FILE_MAP_READ, (DWORD)(qwFileOffset >> 32), (DWORD)(qwFileOffset & 0xFFFFFFFF), dwBytesInBlock); // count the number of 0s in this block. for (DWORD dwByte = 0; dwByte < dwBytesInBlock; dwByte++) { if (pbFile[dwByte] == 'r') qwNumOf0s++; } // unmap the view; we don't want multiple views // in our address space. UnmapViewOfFile(pbFile); // skip to the next set of bytes in the file. qwFileOffset += dwBytesInBlock; qwFileSize -= dwBytesInBlock; } CloseHandle(hFileMapping); return qwNumOf0s; }
如果是往里边写数据就用 memcpy把数据考入pbFile指向的内存。还有就是把读标志该成写标志;