Read File Fast

5 Jun 2005 8:03 PM

Roby Eisenbraun Martins

Hi,

     Which one is the fast way to open a file and read it. If possible
without buffers and any other kind of conversion. I need one step from the
harddisk to the Array.

    Right now I am using FileStream. Is this the best way?

     Thank you,
     Roby Eisenbraun Martins

5 Jun 2005 8:15 PM

Jon Skeet [C# MVP]

Roby Eisenbraun Martins
<RobyEisenbraunMart***@discussions.microsoft.com> wrote:

> Which one is the fast way to open a file and read it. If possible
> without buffers and any other kind of conversion. I need one step from the
> harddisk to the Array.
>
> Right now I am using FileStream. Is this the best way?

There are always going to be buffers at some level, but using
FileStream is likely to be as fast as you'll get in .NET. Do you have
any evidence that file reading is the bottleneck in your application?
Have you worked out how much slower your use of FileStream is than the
theoretical maximum transfer rate of your disk?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

5 Jun 2005 8:32 PM

Willy Denoyette [MVP]

"Roby Eisenbraun Martins" <RobyEisenbraunMart***@discussions.microsoft.com>
wrote in message news:FF6EB5DD-BDAF-47EF-9AD5-017B9BC7AD34@microsoft.com...
>    Hi,
>
>     Which one is the fast way to open a file and read it. If possible
> without buffers and any other kind of conversion. I need one step from the
> harddisk to the Array.
>
>    Right now I am using FileStream. Is this the best way?
>
>     Thank you,
>     Roby Eisenbraun Martins

It will depend on the drive configuration, FileStream goes pretty fast on
single drives, but using striped disk arrays, you get better results when
doing un-buffered IO.
This can be achieved by using PInvoke to call CreateFile with the
FILE_FLAG_NO_BUFFERING flag specified, the handle returned can simply be
passed as argument to one of the FileStream constructors taking an handle to
a File.

Willy.

5 Jun 2005 9:16 PM

Roby Eisenbraun Martins

So is possible, but only using Windows API?

Show quoteHide quote

"Willy Denoyette [MVP]" wrote:

>
> "Roby Eisenbraun Martins" <RobyEisenbraunMart***@discussions.microsoft.com>
> wrote in message news:FF6EB5DD-BDAF-47EF-9AD5-017B9BC7AD34@microsoft.com...
> >    Hi,
> >
> >     Which one is the fast way to open a file and read it. If possible
> > without buffers and any other kind of conversion. I need one step from the
> > harddisk to the Array.
> >
> >    Right now I am using FileStream. Is this the best way?
> >
> >     Thank you,
> >     Roby Eisenbraun Martins
>
> It will depend on the drive configuration, FileStream goes pretty fast on
> single drives, but using striped disk arrays, you get better results when
> doing un-buffered IO.
> This can be achieved by using PInvoke to call CreateFile with the
> FILE_FLAG_NO_BUFFERING flag specified, the handle returned can simply be
> passed as argument to one of the FileStream constructors taking an handle to
> a File.
>
> Willy.
>
>
>
>

5 Jun 2005 9:40 PM

Willy Denoyette [MVP]

"Roby Eisenbraun Martins" <RobyEisenbraunMart***@discussions.microsoft.com>
wrote in message news:0EB8DD12-FC4F-44EC-8A08-E85A599C7AAD@microsoft.com...
> So is possible, but only using Windows API?
>

Yep, but there is only the CreateFile API to "PInvoke", note that the FCL
also uses PInvoke to call the Win32 API's for file IO.
Note also that while you bypass the NTFS filesystem cache and consequently
avoids the overhead of moving data between the FS cache, the .NET cache and
the application space, you are also loosing the advantage of the FS cache
when multiple applications might read the same file or if the application
updates the file data. Some applications, like file copy programs or
databases, might take advantage of un-buffered IO while most others don't
need or want this at all.

Willy.

Show quoteHide quote

> "Willy Denoyette [MVP]" wrote:
>
>>
>> "Roby Eisenbraun Martins"
>> <RobyEisenbraunMart***@discussions.microsoft.com>
>> wrote in message
>> news:FF6EB5DD-BDAF-47EF-9AD5-017B9BC7AD34@microsoft.com...
>> >    Hi,
>> >
>> >     Which one is the fast way to open a file and read it. If possible
>> > without buffers and any other kind of conversion. I need one step from
>> > the
>> > harddisk to the Array.
>> >
>> >    Right now I am using FileStream. Is this the best way?
>> >
>> >     Thank you,
>> >     Roby Eisenbraun Martins
>>
>> It will depend on the drive configuration, FileStream goes pretty fast on
>> single drives, but using striped disk arrays, you get better results when
>> doing un-buffered IO.
>> This can be achieved by using PInvoke to call CreateFile with the
>> FILE_FLAG_NO_BUFFERING flag specified, the handle returned can simply be
>> passed as argument to one of the FileStream constructors taking an handle
>> to
>> a File.
>>
>> Willy.
>>
>>
>>
>>

6 Jun 2005 2:30 PM

Cowboy (Gregory A. Beamer) - MVP

Do you have a performance issue when you are using FileStream?

If the answer is yes, you should search for a faster method, which will
likely entail poking into the Windows API. This will cause a small perf hit
with Interop, but you will likely gain it back with API file manipulation.

If no, streams are very lightweight and fast. I cannot, at present, think of
a faster "all .NET" solution.

--
Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA

***************************
Think Outside the Box!
***************************

Show quoteHide quote

"Roby Eisenbraun Martins" wrote:

>     Hi,
>
>      Which one is the fast way to open a file and read it. If possible
> without buffers and any other kind of conversion. I need one step from the
> harddisk to the Array.
>
>     Right now I am using FileStream. Is this the best way?
>
>      Thank you,
>      Roby Eisenbraun Martins