|
dev
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
How to speed up reading of large disk filesSorry if this is the wrong news group for this post.
I have some large disk files (15-20 MB) that are made up of many records of different types and sizes. I use a BinaryReader to process all the data sequentially (i.e. with many reads operations) and load it into my .NET app. I will have several .NET Apps running on the same computer each of which will need to read these files but never change them. The performance is not awful, but I would like it to be faster. I have looked at Memory File mapping, RAM disks, etc. but I would like to know if someone has a better idea especially if it stays within normal .NET I/O funcationality. Thanks in advance for your help. Valerie Hough "Valerie Hough" <sa***@pcTrans.com> wrote in message The easiest thing you can do to speed up disk i/o is to read larger portions news:eonUedMDHHA.996@TK2MSFTNGP02.phx.gbl... > [...] > The performance is not awful, but I would like it to be faster. I have > looked at Memory File mapping, RAM disks, etc. but I would like to know if > someone has a better idea especially if it stays within normal .NET I/O > funcationality. of the file at a time. With .NET, you either do this explicitly with large reads, or you can explicitly set the FileStream to use buffered i/o when you open the file, and specify a large buffer size. I know it's not a very sophisticated technique, but I've had good success with it. I try to use buffers that are multiples of the default page size (4096 bytes), and have found through experimentation that buffers larger than two or three pages reach a point of diminishing returns. At that point, to get better performance would probably require a lot more work on my part. :) The other thing you can do is try to make sure that you can operate on the data directly from the buffers used to read it. If you stick another copy operation of the data in your code, that will slow you down, especially if you're dealing with a lot of data. My apologies if the above suggestions seem obvious and not useful. :) Pete |
|||||||||||||||||||||||