High throughput disk write: CreateFile/WriteFile?
Hello all,
I have an application where I need to write text and binary (numeric)
data to a file at ~200 MB/s (8MB/40ms). I am currently using fprintf
and fwrite, respectively, but I'm not achieving the desired
throughput. I've been looking into the Windows CreateFile based
methods instead.
I am using a pcix hardware raid adapter with RAID0 across 4 drives. I
have benchmarked my system using h2benchw v3.6 with good results. For
my desired file sizes, anywhere from 8MB to 64MB (I'm flexible on how
many 8MB data sets are stored in a single file), I am able to achieve
up to 275MB/s.
I'm hoping it's possible to get close to that using alternatives from
the standard C options. Any suggestions?
For now I'm trying CreateFile/WriteFile, without success. My code
looks like this:
<SNIP>
HANDLE hWriteFile = NULL; // File handle
LPDWORD lpNumBytesWritten = NULL; // Number of bytes written
(WriteFile)
char szTextToAppendToLog[1024]; // Temp char buffer
char szFilePath[1024]; // Output file name
char szTimeStamp[32]; // Time character string
char szTemp[128]; // Temp character string
// Open the output file for write.
hWriteFile = CreateFile(
szFilePath, // File path
GENERIC_WRITE, // Open for write
NULL, // Do not share
NULL, // Default security
CREATE_ALWAYS, // Overwrite existing files
FILE_FLAG_WRITE_THROUGH,//FILE_FLAG_OVERLAPPED, //
Normal file
NULL); // No template
if (hWriteFile == INVALID_HANDLE_VALUE)
{
sprintf_s(szTextToAppendToLog,sizeof(szTextToAppendToLog),
"ERROR: Output file %s failed to open.",
szFilePath);
pThis->UI->AppendToStatLog(szTextToAppendToLog);
}
....
// Write ASCII header to file.
sprintf_s(szTemp, sizeof(szTemp), "MyData: Date:%s\r
\n",szTimeStamp);
WriteFile(
hWriteFile,
szTemp,
(DWORD) sizeof(szTemp),
lpNumBytesWritten,
NULL);
</SNIP>
As soon as I hit WriteFile(...) my application hits an unhandled
exception:
"Unhandled exception at 0x7c810e0c in DataCapture.exe: 0xC0000005:
Access violation writing location 0x00000000."
I'm guessing I'm not using CreateFile correctly?
Thanks,
-Brandon