Home All Groups Group Topic Archive Search About

Limit to filesize on Streamwriter?

Author
12 Jun 2006 12:26 PM
eric.goforth
Hello,

I have a VB.NET app written in VS 2005 that's copying and then deleting
a very large number of files (250,000).  It appears to run
successfully, but I don't see it finish in it's log file.  The end of
the logfile looks like:

C:\Documents and Settings\MyUser\My Documents\Visual Studio
2005\Projects\
MyUtility\bin\Debug>tail log.txt
Deleting d:\attachments\sa6A8F.tmp
Deleting d:\attachments\sa6A92.tmp
Deleting d:\attachments\sa73EF.tmp
Deleting d:\attachments\sa73F2.tmp
Deleting d:\attachments\sa73F5.tmp
Deleting d:\attachments\sa73FC.tmp
Deleting d:\attachments\sa73FF.tmp
Deleting d:\attachments\sa7402.tmp
Deleting d:\attachments\sa8BC5.tmp
Deleting d:\attachments\zia03444
Deleting d:\attachm
C:\Documents and Settings\MyUser\My Documents\Visual Studio
2005\Projects\
MyUtility\bin\Debug>


The log file is 27,602,944 bytes.  I'm writing to the logfile like so:

Dim swErrLog As StreamWriter = New
StreamWriter(AppDomain.CurrentDomain.BaseDirectory & "\Log.txt")

swErrLog.WriteLine("Deleting " & aFiles(i))

Is there some upper limit to what size file a StreamWriter can handle?

Thanks,
Eric

Author
12 Jun 2006 2:11 PM
Vadym Stetsyak
Hello, eric.gofo***@gmail.com!

eg> I have a VB.NET app written in VS 2005 that's copying and then deleting
eg> a very large number of files (250,000).  It appears to run
eg> successfully, but I don't see it finish in it's log file.  The end of
eg> the logfile looks like:

eg> C:\Documents and Settings\MyUser\My Documents\Visual Studio
eg> 2005\Projects\
eg> MyUtility\bin\Debug>tail log.txt
eg> Deleting d:\attachments\sa6A8F.tmp
eg> Deleting d:\attachments\sa6A92.tmp
eg> Deleting d:\attachments\sa73EF.tmp
eg> Deleting d:\attachments\sa73F2.tmp
eg> Deleting d:\attachments\sa73F5.tmp
eg> Deleting d:\attachments\sa73FC.tmp
eg> Deleting d:\attachments\sa73FF.tmp
eg> Deleting d:\attachments\sa7402.tmp
eg> Deleting d:\attachments\sa8BC5.tmp
eg> Deleting d:\attachments\zia03444
eg> Deleting d:\attachm
eg> C:\Documents and Settings\MyUser\My Documents\Visual Studio
eg> 2005\Projects\
eg> MyUtility\bin\Debug>

eg> The log file is 27,602,944 bytes.  I'm writing to the logfile like so:

eg> Dim swErrLog As StreamWriter = New
eg> StreamWriter(AppDomain.CurrentDomain.BaseDirectory & "\Log.txt")

eg> swErrLog.WriteLine("Deleting " & aFiles(i))

eg> Is there some upper limit to what size file a StreamWriter can handle?

Try calling Flush() method to make stream write data to the file.

Generally, data written to the file is cached and if you will call Flush(), cached data will be written to the disk.

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com
Author
12 Jun 2006 3:06 PM
Carl Daniel [VC++ MVP]
Vadym Stetsyak wrote:
> Hello, eric.gofo***@gmail.com!
> eg> Is there some upper limit to what size file a StreamWriter can
> handle?
>
> Try calling Flush() method to make stream write data to the file.
>
> Generally, data written to the file is cached and if you will call
> Flush(), cached data will be written to the disk.

Or better yet, Dispose the stream - which will internally call Flush and
will release the OS file handle as well.

using (Stream os =  new FileStream( ... ))
{
    // do your thing, writing to os
}

// os.Dispose is called, flushing the stream and closing the file

-cd
Author
12 Jun 2006 6:12 PM
eric.goforth
Carl Daniel [VC++ MVP] wrote:
Show quote
> Vadym Stetsyak wrote:
> > Hello, eric.gofo***@gmail.com!
> > eg> Is there some upper limit to what size file a StreamWriter can
> > handle?
> >
> > Try calling Flush() method to make stream write data to the file.
> >
> > Generally, data written to the file is cached and if you will call
> > Flush(), cached data will be written to the disk.
>
> Or better yet, Dispose the stream - which will internally call Flush and
> will release the OS file handle as well.
>
> using (Stream os =  new FileStream( ... ))
> {
>     // do your thing, writing to os
> }
>
> // os.Dispose is called, flushing the stream and closing the file
>
> -cd

Hello,

I have my streamwriter declared at the module level and them I'm doing
my swErrLog.WriteLine("Deleting " & aFiles(i)) in various subroutines
and functions in my module.

Is this the best way to do this?  I figured that it would be better to
have everyone share one Streamwriter object than reinstantiating it
tens of thousands of times.  If I call a Dispose won't it destroy my
Streamwriter object?

-Eric
Author
12 Jun 2006 6:19 PM
Michael D. Ober
Is this a log file?

Mike Ober.

<eric.gofo***@gmail.com> wrote in message
Show quote
news:1150135979.176746.73080@f6g2000cwb.googlegroups.com...
>
> Carl Daniel [VC++ MVP] wrote:
> > Vadym Stetsyak wrote:
> > > Hello, eric.gofo***@gmail.com!
> > > eg> Is there some upper limit to what size file a StreamWriter can
> > > handle?
> > >
> > > Try calling Flush() method to make stream write data to the file.
> > >
> > > Generally, data written to the file is cached and if you will call
> > > Flush(), cached data will be written to the disk.
> >
> > Or better yet, Dispose the stream - which will internally call Flush and
> > will release the OS file handle as well.
> >
> > using (Stream os =  new FileStream( ... ))
> > {
> >     // do your thing, writing to os
> > }
> >
> > // os.Dispose is called, flushing the stream and closing the file
> >
> > -cd
>
> Hello,
>
> I have my streamwriter declared at the module level and them I'm doing
> my swErrLog.WriteLine("Deleting " & aFiles(i)) in various subroutines
> and functions in my module.
>
> Is this the best way to do this?  I figured that it would be better to
> have everyone share one Streamwriter object than reinstantiating it
> tens of thousands of times.  If I call a Dispose won't it destroy my
> Streamwriter object?
>
> -Eric
>
>
Author
12 Jun 2006 6:24 PM
eric.goforth
Michael D. Ober wrote:
> Is this a log file?
>
> Mike Ober.
>

Yes
Author
12 Jun 2006 6:56 PM
Michael D. Ober
Eric,

Here's a complete logfile class written in VB 2005.

Option Explicit On
Option Compare Text
Option Strict On

Module LogInstance
   Public logs as New LogFile
End Module

Public Class LogFile
    Implements IDisposable

    Private BaseFile As String = "" ' Contains all but the log file date and
extension
    Private FileName As String = "" ' Contains the actual file that is being
written
    Private Opened As Date ' When did this log file open
    Private Const LogFileEntryDTStamp As String = "yyyyMMdd HH.mm.ss.fff"
    Private Const LogFileNameDateStamp As String = "yyyyMMdd"
    Private ts As StreamWriter ' Internal file control

    ' Protect multiple threads performing simultaneous writes to this log
file
    Private LockWriteMessage As New System.Threading.SynchronizationContext
    Public UseVersion As Boolean = True ' Controls version portion of file
name
    Public Sub New()
    End Sub
    Public Sub New(ByVal fName As String)
        Me.new()
        Name = fName
    End Sub

    Public Property Name() As String
        Get
            Return FileName
        End Get
        Set(ByVal FileBase As String)
            ' Parse the FileBase into the path and base part of the
filename. Usually, FileBase will be the application name
            If InStr(FileBase, "\") = 0 Then
                BaseFile = Environ$("LogDir")                   ' In our
corporate environment, the Domain defines this via Group Policy
                If BaseFile = "" Then BaseFile = "c:"
                BaseFile = TrailSlash(BaseFile) & My.Application.Info.Title
            Else
                BaseFile = Left$(FileBase, InStrRev(FileBase, "\") - 1)
                FileBase = Mid$(FileBase, InStrRev(FileBase, "\") + 1)
            End If

            ' BaseFile now contains the directory; ensure it exists
            If Not My.Computer.FileSystem.DirectoryExists(BaseFile) Then
My.Computer.FileSystem.CreateDirectory(BaseFile)
            ' Check for versioning and add it if required
            If UseVersion AndAlso InStr(FileBase, " v.") = 0 Then FileBase
+= " v." & My.Application.Info.Version.ToString()

#If DEBUG Then
            FileBase += " - DEBUG "
#End If

            ' Set basefile to be the entire file except for the date; this
will potentially change on every open
            BaseFile = TrailSlash(BaseFile) & FileBase & " - " &
UCase$(Environ$("USERNAME")) & " on " & UCase$(My.Computer.Name) + " - "
            ' FileName will be the complete log file name
            FileName = BaseFile & Today.ToString(LogFileNameDateStamp) &
".log"

            ' Open it and set the opened date to today; used by ReOpen
            ts = My.Computer.FileSystem.OpenTextFileWriter(FileName, True,
System.Text.Encoding.ASCII)
            Opened = Today
            ' Log the fact that we opened the log file
            WriteLog(TypeName(Me) & " '" & FileName & "' Opened")
        End Set
    End Property

    Public Sub WriteLog(Optional ByVal Message As String = "")
        ' Remove trailing spaces and tabs as they are simply wasted file
space
        Message = Message.TrimEnd(" "c, CChar(vbTab), CChar(vbBack),
CChar(vbCr), CChar(vbLf), CChar(vbFormFeed), CChar(vbNullChar),
CChar(vbVerticalTab))
        ' All writes to the logfile must complete before another thread can
start.
        SyncLock LockWriteMessage
            If FileName = "" Then Name = My.Application.Info.Title Else
ReOpen()
            If Message <> "" Then
                ts.WriteLine(Now.ToString(LogFileEntryDTStamp) & vbTab &
Message)
                ts.Flush()
            Else
                ' Since there is no reason to flush empty lines, I chose not
to set the AutoFlush property to True.
                ts.WriteLine("")
            End If
        End SyncLock
    End Sub

    ' Check for change of day.  This way the logfile for this application
will reopen automatically the first time it is written to on a new day.
    ' A side effect of this is that if you start and stop the same program
multiple times in the same day, the logfile will be appended to during the
day.
    ' This is why I implemented the IDisposable interface.
    Public Sub ReOpen()
        If FileName <> "" AndAlso Opened.Date <> Today Then
            SyncLock LockWriteMessage
                Opened = Today
                FileName = BaseFile & Today.ToString(LogFileNameDateStamp) &
".log"
                WriteLog("Day Change Closure")
                WriteLog("Log File Closing; new logfile '" & FileName & "'
being opened")
                ts.Close()
                ts = My.Computer.FileSystem.OpenTextFileWriter(FileName,
True, System.Text.Encoding.ASCII)
                WriteLog(TypeName(Me) & " '" & FileName & "' Opened after
Day Change")
            End SyncLock
        End If
    End Sub

    Public Function Close() As String
        Dim fName As String = FileName
        If fName <> "" Then ts.Close()
        FileName = ""
        Return fName
    End Function

#Region " IDisposable Support "
    Private disposedValue As Boolean = False ' To detect redundant calls
    ' IDisposable
    Protected Overridable Sub Dispose(ByVal disposing As Boolean)
        Const msgDisposing As String = "Logfile being Garbage Collected"
        If FileName <> "" Then
            Try
                WriteLog(msgDisposing)
                Close()
            Catch ex As Exception
                ' The framework GC tends to dispose the internal
streamwriter before the class IDisposable interface is called.  If you don't
care about this final message
                ' you can dump the entire IDispose interface.
                SyncLock LockWriteMessage
                    Dim tsClosing As StreamWriter =
My.Computer.FileSystem.OpenTextFileWriter(FileName, True,
System.Text.Encoding.ASCII)
                    tsClosing.WriteLine(Now.ToString(LogFileEntryDTStamp) &
vbTab & msgDisposing)
                    tsClosing.Close()
                End SyncLock
            End Try
        End If
        Me.disposedValue = True
    End Sub

    ' This code added by Visual Basic to correctly implement the disposable
pattern.
    Public Sub Dispose() Implements IDisposable.Dispose
    ' Do not change this code. Put cleanup code in Dispose(ByVal disposing
As Boolean) above.
         Dispose(True)
         GC.SuppressFinalize(Me)
    End Sub

    Protected Overrides Sub Finalize()
        On Error Resume Next     ' We don't really care that there is an
error, but we don't want this to crash either
        Dispose(False)
        ts.Dispose()
    End Sub
#End Region
End Class

<eric.gofo***@gmail.com> wrote in message
Show quote
news:1150136640.325277.231940@u72g2000cwu.googlegroups.com...
>
> Michael D. Ober wrote:
> > Is this a log file?
> >
> > Mike Ober.
> >
>
> Yes
>
>

AddThis Social Bookmark Button