[Net 2.0] question about background worker and thread synchronization

6 Oct 2006 10:58 AM

Steve B.

Hi,

I'm building an application that performs some long operations against
files.
In order to view the progress, I'm using two background workers.

The first backgroundworker is listing recursively all files in a folder and
each files are queued into a Queue<FileInfo> object.
I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
because lisiting all files is quite long and I want to start my process as
soon as on file is found.

The second backgroundworker will execute the process agains each found
files.

I'm wondering how can I properly code my application in order to run the
second background worker only when files are filled. Note that I must run
the background worker from the window thread because the progressedchanged
must change controls properties.

Since the two background worker are started simultaneously, I need a way to
block the second BGWorker until the first file has been found. I using a
while clause but I'm not sure this is a good practice regarding the
performances.

I'm using by now this code snippet :

public partial class Form1 : Form
{
/// <summary>
/// Initializes a new instance of the <see cref="T:Form1"/> class.
/// </summary>
public Form1()
{
InitializeComponent();
m_filesToAnalyse = new Queue<FileInfo>();
m_filesToCompare = new Queue<FileScanResult>();
}
private Queue<FileInfo> m_filesToAnalyse;
private bool m_remainingFiles;
private void button1_Click(object sender, EventArgs e)
{
m_remainingFiles = true;
folderBrowsing.RunWorkerAsync(@"I:\Formations");
filesAnalysing.RunWorkerAsync();
}
private void AnalyseFolder(DirectoryInfo dir) // Used for recursive browsing
{
//Thread.Sleep(50);
folderBrowsing.ReportProgress(0, dir);
Debug.WriteLine("* " + dir.FullName);
foreach (FileInfo file in dir.GetFiles())
{
m_filesToAnalyse.Enqueue(file);
}
foreach (DirectoryInfo subDir in dir.GetDirectories())
{
AnalyseFolder(subDir);
}
}
#region folderBrowsing
private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
{
string folder = e.Argument.ToString();
DirectoryInfo dir = new DirectoryInfo(folder);
AnalyseFolder(dir);
}
private void folderBrowsing_ProgressChanged(object sender,
ProgressChangedEventArgs e)
{
listBox1.Items.Insert(0, string.Format(
"Folder : {0}",
((DirectoryInfo)e.UserState).FullName
));
}
private void folderBrowsing_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
m_remainingFiles = false;
listBox1.Items.Insert(0, "parcours fichiers fini");
}
#endregion
#region filesAnalysing
private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
{
lock (m_filesToAnalyse)
{
while (m_remainingFiles || m_filesToAnalyse.Count > 0)
{
if(m_filesToAnalyse.Count > 0)
{
FileInfo fi = m_filesToAnalyse.Dequeue();
filesAnalysing.ReportProgress(0, fi);
Debug.WriteLine("- " + fi.FullName);
/*
* Perform here the custom process
*/

);
}
}
}
}
private void filesAnalysing_ProgressChanged(object sender,
ProgressChangedEventArgs e)
{
listBox1.Items.Insert(0, string.Format(
"Fichier : {0}",
((FileInfo)e.UserState).FullName
));
}
private void filesAnalysing_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
listBox1.Items.Insert(0, "analyse fichiers fini");
}
#endregion
}

Am I on the right way ?

Thanks in advance,
Steve

6 Oct 2006 11:58 AM

Dmytro Lapshyn [MVP]

Hi Steve,

1) Cross-thread synchronization is usually done by using so-called
synchronization ojects. Consider AutoResetEvent or ManualResetEvent. Using
while() loops will make the waiting thread consume CPU cycles which is
obviously bad.

2) You don't need to run a worker on a UI thread in order to update the UI.
Consider Control.Invoke to marshal UI update requests to the main UI thread.

--
Regards,
Dmytro Lapshyn [MVP]
http://blogs.vbcity.com/DmytroL

Show quote

"Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
news:OKwKqZT6GHA.4476@TK2MSFTNGP04.phx.gbl...
> Hi,
>
> I'm building an application that performs some long operations against
> files.
> In order to view the progress, I'm using two background workers.
>
> The first backgroundworker is listing recursively all files in a folder
> and each files are queued into a Queue<FileInfo> object.
> I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
> because lisiting all files is quite long and I want to start my process as
> soon as on file is found.
>
> The second backgroundworker will execute the process agains each found
> files.
>
> I'm wondering how can I properly code my application in order to run the
> second background worker only when files are filled. Note that I must run
> the background worker from the window thread because the progressedchanged
> must change controls properties.
>
> Since the two background worker are started simultaneously, I need a way
> to block the second BGWorker until the first file has been found. I using
> a while clause but I'm not sure this is a good practice regarding the
> performances.
>
> I'm using by now this code snippet :
>
> public partial class Form1 : Form
> {
> /// <summary>
> /// Initializes a new instance of the <see cref="T:Form1"/> class.
> /// </summary>
> public Form1()
> {
> InitializeComponent();
> m_filesToAnalyse = new Queue<FileInfo>();
> m_filesToCompare = new Queue<FileScanResult>();
> }
> private Queue<FileInfo> m_filesToAnalyse;
> private bool m_remainingFiles;
> private void button1_Click(object sender, EventArgs e)
> {
> m_remainingFiles = true;
> folderBrowsing.RunWorkerAsync(@"I:\Formations");
> filesAnalysing.RunWorkerAsync();
> }
> private void AnalyseFolder(DirectoryInfo dir) // Used for recursive
> browsing
> {
> //Thread.Sleep(50);
> folderBrowsing.ReportProgress(0, dir);
> Debug.WriteLine("* " + dir.FullName);
> foreach (FileInfo file in dir.GetFiles())
> {
> m_filesToAnalyse.Enqueue(file);
> }
> foreach (DirectoryInfo subDir in dir.GetDirectories())
> {
> AnalyseFolder(subDir);
> }
> }
> #region folderBrowsing
> private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
> {
> string folder = e.Argument.ToString();
> DirectoryInfo dir = new DirectoryInfo(folder);
> AnalyseFolder(dir);
> }
> private void folderBrowsing_ProgressChanged(object sender,
> ProgressChangedEventArgs e)
> {
> listBox1.Items.Insert(0, string.Format(
> "Folder : {0}",
> ((DirectoryInfo)e.UserState).FullName
> ));
> }
> private void folderBrowsing_RunWorkerCompleted(object sender,
> RunWorkerCompletedEventArgs e)
> {
> m_remainingFiles = false;
> listBox1.Items.Insert(0, "parcours fichiers fini");
> }
> #endregion
> #region filesAnalysing
> private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
> {
> lock (m_filesToAnalyse)
> {
> while (m_remainingFiles || m_filesToAnalyse.Count > 0)
> {
> if(m_filesToAnalyse.Count > 0)
> {
> FileInfo fi = m_filesToAnalyse.Dequeue();
> filesAnalysing.ReportProgress(0, fi);
> Debug.WriteLine("- " + fi.FullName);
> /*
> * Perform here the custom process
> */
>
> );
> }
> }
> }
> }
> private void filesAnalysing_ProgressChanged(object sender,
> ProgressChangedEventArgs e)
> {
> listBox1.Items.Insert(0, string.Format(
> "Fichier : {0}",
> ((FileInfo)e.UserState).FullName
> ));
> }
> private void filesAnalysing_RunWorkerCompleted(object sender,
> RunWorkerCompletedEventArgs e)
> {
> listBox1.Items.Insert(0, "analyse fichiers fini");
> }
> #endregion
> }
>
> Am I on the right way ?
>
> Thanks in advance,
> Steve
>

6 Oct 2006 12:19 PM

Steve B.

1. What is the difference between the two objects ?

2. I use the background worker in order to avoid invoking on controls. I
found the code is easier to read with progressed changed event that obscur
Control.Invoke, that's the goal of the background worker...

Steve

"Dmytro Lapshyn [MVP]" <x-code@no-spam-please.hotpop.com> a ecrit dans le

message de news: ejhbC7T6GHA.4***@TK2MSFTNGP03.phx.gbl...
Show quote

> Hi Steve,
>
> 1) Cross-thread synchronization is usually done by using so-called
> synchronization ojects. Consider AutoResetEvent or ManualResetEvent. Using
> while() loops will make the waiting thread consume CPU cycles which is
> obviously bad.
>
> 2) You don't need to run a worker on a UI thread in order to update the
> UI. Consider Control.Invoke to marshal UI update requests to the main UI
> thread.
>
> --
> Regards,
> Dmytro Lapshyn [MVP]
> http://blogs.vbcity.com/DmytroL
>
> "Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
> news:OKwKqZT6GHA.4476@TK2MSFTNGP04.phx.gbl...
>> Hi,
>>
>> I'm building an application that performs some long operations against
>> files.
>> In order to view the progress, I'm using two background workers.
>>
>> The first backgroundworker is listing recursively all files in a folder
>> and each files are queued into a Queue<FileInfo> object.
>> I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
>> because lisiting all files is quite long and I want to start my process
>> as soon as on file is found.
>>
>> The second backgroundworker will execute the process agains each found
>> files.
>>
>> I'm wondering how can I properly code my application in order to run the
>> second background worker only when files are filled. Note that I must run
>> the background worker from the window thread because the
>> progressedchanged must change controls properties.
>>
>> Since the two background worker are started simultaneously, I need a way
>> to block the second BGWorker until the first file has been found. I using
>> a while clause but I'm not sure this is a good practice regarding the
>> performances.
>>
>> I'm using by now this code snippet :
>>
>> public partial class Form1 : Form
>> {
>> /// <summary>
>> /// Initializes a new instance of the <see cref="T:Form1"/> class.
>> /// </summary>
>> public Form1()
>> {
>> InitializeComponent();
>> m_filesToAnalyse = new Queue<FileInfo>();
>> m_filesToCompare = new Queue<FileScanResult>();
>> }
>> private Queue<FileInfo> m_filesToAnalyse;
>> private bool m_remainingFiles;
>> private void button1_Click(object sender, EventArgs e)
>> {
>> m_remainingFiles = true;
>> folderBrowsing.RunWorkerAsync(@"I:\Formations");
>> filesAnalysing.RunWorkerAsync();
>> }
>> private void AnalyseFolder(DirectoryInfo dir) // Used for recursive
>> browsing
>> {
>> //Thread.Sleep(50);
>> folderBrowsing.ReportProgress(0, dir);
>> Debug.WriteLine("* " + dir.FullName);
>> foreach (FileInfo file in dir.GetFiles())
>> {
>> m_filesToAnalyse.Enqueue(file);
>> }
>> foreach (DirectoryInfo subDir in dir.GetDirectories())
>> {
>> AnalyseFolder(subDir);
>> }
>> }
>> #region folderBrowsing
>> private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
>> {
>> string folder = e.Argument.ToString();
>> DirectoryInfo dir = new DirectoryInfo(folder);
>> AnalyseFolder(dir);
>> }
>> private void folderBrowsing_ProgressChanged(object sender,
>> ProgressChangedEventArgs e)
>> {
>> listBox1.Items.Insert(0, string.Format(
>> "Folder : {0}",
>> ((DirectoryInfo)e.UserState).FullName
>> ));
>> }
>> private void folderBrowsing_RunWorkerCompleted(object sender,
>> RunWorkerCompletedEventArgs e)
>> {
>> m_remainingFiles = false;
>> listBox1.Items.Insert(0, "parcours fichiers fini");
>> }
>> #endregion
>> #region filesAnalysing
>> private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
>> {
>> lock (m_filesToAnalyse)
>> {
>> while (m_remainingFiles || m_filesToAnalyse.Count > 0)
>> {
>> if(m_filesToAnalyse.Count > 0)
>> {
>> FileInfo fi = m_filesToAnalyse.Dequeue();
>> filesAnalysing.ReportProgress(0, fi);
>> Debug.WriteLine("- " + fi.FullName);
>> /*
>> * Perform here the custom process
>> */
>>
>> );
>> }
>> }
>> }
>> }
>> private void filesAnalysing_ProgressChanged(object sender,
>> ProgressChangedEventArgs e)
>> {
>> listBox1.Items.Insert(0, string.Format(
>> "Fichier : {0}",
>> ((FileInfo)e.UserState).FullName
>> ));
>> }
>> private void filesAnalysing_RunWorkerCompleted(object sender,
>> RunWorkerCompletedEventArgs e)
>> {
>> listBox1.Items.Insert(0, "analyse fichiers fini");
>> }
>> #endregion
>> }
>>
>> Am I on the right way ?
>>
>> Thanks in advance,
>> Steve
>>
>

6 Oct 2006 1:21 PM

Michael D. Ober

Instead of using a straight queue, create a new class that inherits from the
queue class. Then override the add method to start your background method
when a file is added to the queue. You'll need to make the underlying queue
thread safe by using synclock ... end synclock statements (simplest and
sufficient) or reader/writer locks. If you only want a single thread doing
the actual file processing, you'll also need to include a thread object in
the derived class to track the lifetime of your worker thread.

Here's a sample shell to get you started.

imports system.io
imports system.threading

Public Class FileProcessor
   inherits system.collections.queue

   private th as new thread(addressof FileProcessor)
   private so as new syncronizationcontext

   private Shadows sub Add(byval File as string)       ' I'm using Shadows
to block all queue.add methods
      synclock so
         mybase.add file
      end synclock

      if th.isAlive then exit sub
      th.start
   end sub

   private sub FileProcessor()
      do while not queue.isempty
          synclock so
              dim File as string = queue.dequeue()
          end synclock

          ' Process your file
      loop
   end sub

end Class

Mike Ober

Show quote

"Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
news:OKwKqZT6GHA.4476@TK2MSFTNGP04.phx.gbl...
> Hi,
>
> I'm building an application that performs some long operations against
> files.
> In order to view the progress, I'm using two background workers.
>
> The first backgroundworker is listing recursively all files in a folder
> and each files are queued into a Queue<FileInfo> object.
> I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
> because lisiting all files is quite long and I want to start my process as
> soon as on file is found.
>
> The second backgroundworker will execute the process agains each found
> files.
>
> I'm wondering how can I properly code my application in order to run the
> second background worker only when files are filled. Note that I must run
> the background worker from the window thread because the progressedchanged
> must change controls properties.
>
> Since the two background worker are started simultaneously, I need a way
> to block the second BGWorker until the first file has been found. I using
> a while clause but I'm not sure this is a good practice regarding the
> performances.
>
> I'm using by now this code snippet :
>
> public partial class Form1 : Form
> {
> /// <summary>
> /// Initializes a new instance of the <see cref="T:Form1"/> class.
> /// </summary>
> public Form1()
> {
> InitializeComponent();
> m_filesToAnalyse = new Queue<FileInfo>();
> m_filesToCompare = new Queue<FileScanResult>();
> }
> private Queue<FileInfo> m_filesToAnalyse;
> private bool m_remainingFiles;
> private void button1_Click(object sender, EventArgs e)
> {
> m_remainingFiles = true;
> folderBrowsing.RunWorkerAsync(@"I:\Formations");
> filesAnalysing.RunWorkerAsync();
> }
> private void AnalyseFolder(DirectoryInfo dir) // Used for recursive
> browsing
> {
> //Thread.Sleep(50);
> folderBrowsing.ReportProgress(0, dir);
> Debug.WriteLine("* " + dir.FullName);
> foreach (FileInfo file in dir.GetFiles())
> {
> m_filesToAnalyse.Enqueue(file);
> }
> foreach (DirectoryInfo subDir in dir.GetDirectories())
> {
> AnalyseFolder(subDir);
> }
> }
> #region folderBrowsing
> private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
> {
> string folder = e.Argument.ToString();
> DirectoryInfo dir = new DirectoryInfo(folder);
> AnalyseFolder(dir);
> }
> private void folderBrowsing_ProgressChanged(object sender,
> ProgressChangedEventArgs e)
> {
> listBox1.Items.Insert(0, string.Format(
> "Folder : {0}",
> ((DirectoryInfo)e.UserState).FullName
> ));
> }
> private void folderBrowsing_RunWorkerCompleted(object sender,
> RunWorkerCompletedEventArgs e)
> {
> m_remainingFiles = false;
> listBox1.Items.Insert(0, "parcours fichiers fini");
> }
> #endregion
> #region filesAnalysing
> private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
> {
> lock (m_filesToAnalyse)
> {
> while (m_remainingFiles || m_filesToAnalyse.Count > 0)
> {
> if(m_filesToAnalyse.Count > 0)
> {
> FileInfo fi = m_filesToAnalyse.Dequeue();
> filesAnalysing.ReportProgress(0, fi);
> Debug.WriteLine("- " + fi.FullName);
> /*
> * Perform here the custom process
> */
>
> );
> }
> }
> }
> }
> private void filesAnalysing_ProgressChanged(object sender,
> ProgressChangedEventArgs e)
> {
> listBox1.Items.Insert(0, string.Format(
> "Fichier : {0}",
> ((FileInfo)e.UserState).FullName
> ));
> }
> private void filesAnalysing_RunWorkerCompleted(object sender,
> RunWorkerCompletedEventArgs e)
> {
> listBox1.Items.Insert(0, "analyse fichiers fini");
> }
> #endregion
> }
>
> Am I on the right way ?
>
> Thanks in advance,
> Steve
>

6 Oct 2006 3:23 PM

Steve B.

Your solution looks nice. I'm thinknig this could allow me to run multiples
instances of this class simultaneously (to enable object pooling why not).
However, if I separate this process into a clean class w/o any UI, how can I
"reconnect" to the thread that host controls life time ?
I won't be able to call control.invoke unless I pass the reference of the
control... but this would mean that the class can only be used in windows
app...

Steve

"Michael D. Ober" <ober***@.alum.mit.edu.nospam> a écrit dans le message de

news: OgtUMpU6GHA.4***@TK2MSFTNGP03.phx.gbl...
Show quote

> Instead of using a straight queue, create a new class that inherits from
> the queue class. Then override the add method to start your background
> method when a file is added to the queue. You'll need to make the
> underlying queue thread safe by using synclock ... end synclock statements
> (simplest and sufficient) or reader/writer locks. If you only want a
> single thread doing the actual file processing, you'll also need to
> include a thread object in the derived class to track the lifetime of your
> worker thread.
>
> Here's a sample shell to get you started.
>
> imports system.io
> imports system.threading
>
> Public Class FileProcessor
>   inherits system.collections.queue
>
>   private th as new thread(addressof FileProcessor)
>   private so as new syncronizationcontext
>
>   private Shadows sub Add(byval File as string)       ' I'm using Shadows
> to block all queue.add methods
>      synclock so
>         mybase.add file
>      end synclock
>
>      if th.isAlive then exit sub
>      th.start
>   end sub
>
>   private sub FileProcessor()
>      do while not queue.isempty
>          synclock so
>              dim File as string = queue.dequeue()
>          end synclock
>
>          ' Process your file
>      loop
>   end sub
>
> end Class
>
> Mike Ober
>
>
> "Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
> news:OKwKqZT6GHA.4476@TK2MSFTNGP04.phx.gbl...
>> Hi,
>>
>> I'm building an application that performs some long operations against
>> files.
>> In order to view the progress, I'm using two background workers.
>>
>> The first backgroundworker is listing recursively all files in a folder
>> and each files are queued into a Queue<FileInfo> object.
>> I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
>> because lisiting all files is quite long and I want to start my process
>> as soon as on file is found.
>>
>> The second backgroundworker will execute the process agains each found
>> files.
>>
>> I'm wondering how can I properly code my application in order to run the
>> second background worker only when files are filled. Note that I must run
>> the background worker from the window thread because the
>> progressedchanged must change controls properties.
>>
>> Since the two background worker are started simultaneously, I need a way
>> to block the second BGWorker until the first file has been found. I using
>> a while clause but I'm not sure this is a good practice regarding the
>> performances.
>>
>> I'm using by now this code snippet :
>>
>> public partial class Form1 : Form
>> {
>> /// <summary>
>> /// Initializes a new instance of the <see cref="T:Form1"/> class.
>> /// </summary>
>> public Form1()
>> {
>> InitializeComponent();
>> m_filesToAnalyse = new Queue<FileInfo>();
>> m_filesToCompare = new Queue<FileScanResult>();
>> }
>> private Queue<FileInfo> m_filesToAnalyse;
>> private bool m_remainingFiles;
>> private void button1_Click(object sender, EventArgs e)
>> {
>> m_remainingFiles = true;
>> folderBrowsing.RunWorkerAsync(@"I:\Formations");
>> filesAnalysing.RunWorkerAsync();
>> }
>> private void AnalyseFolder(DirectoryInfo dir) // Used for recursive
>> browsing
>> {
>> //Thread.Sleep(50);
>> folderBrowsing.ReportProgress(0, dir);
>> Debug.WriteLine("* " + dir.FullName);
>> foreach (FileInfo file in dir.GetFiles())
>> {
>> m_filesToAnalyse.Enqueue(file);
>> }
>> foreach (DirectoryInfo subDir in dir.GetDirectories())
>> {
>> AnalyseFolder(subDir);
>> }
>> }
>> #region folderBrowsing
>> private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
>> {
>> string folder = e.Argument.ToString();
>> DirectoryInfo dir = new DirectoryInfo(folder);
>> AnalyseFolder(dir);
>> }
>> private void folderBrowsing_ProgressChanged(object sender,
>> ProgressChangedEventArgs e)
>> {
>> listBox1.Items.Insert(0, string.Format(
>> "Folder : {0}",
>> ((DirectoryInfo)e.UserState).FullName
>> ));
>> }
>> private void folderBrowsing_RunWorkerCompleted(object sender,
>> RunWorkerCompletedEventArgs e)
>> {
>> m_remainingFiles = false;
>> listBox1.Items.Insert(0, "parcours fichiers fini");
>> }
>> #endregion
>> #region filesAnalysing
>> private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
>> {
>> lock (m_filesToAnalyse)
>> {
>> while (m_remainingFiles || m_filesToAnalyse.Count > 0)
>> {
>> if(m_filesToAnalyse.Count > 0)
>> {
>> FileInfo fi = m_filesToAnalyse.Dequeue();
>> filesAnalysing.ReportProgress(0, fi);
>> Debug.WriteLine("- " + fi.FullName);
>> /*
>> * Perform here the custom process
>> */
>>
>> );
>> }
>> }
>> }
>> }
>> private void filesAnalysing_ProgressChanged(object sender,
>> ProgressChangedEventArgs e)
>> {
>> listBox1.Items.Insert(0, string.Format(
>> "Fichier : {0}",
>> ((FileInfo)e.UserState).FullName
>> ));
>> }
>> private void filesAnalysing_RunWorkerCompleted(object sender,
>> RunWorkerCompletedEventArgs e)
>> {
>> listBox1.Items.Insert(0, "analyse fichiers fini");
>> }
>> #endregion
>> }
>>
>> Am I on the right way ?
>>
>> Thanks in advance,
>> Steve
>>
>
>

7 Oct 2006 1:01 AM

Michael D. Ober

I actually use it in a console application. Any thread can write to the
console safely. If you need to access the worker thread, make the thread
available via the class interface. I would abstract the thread operations
so that the class has the control over how the thread operates.

Mike Ober.

Show quote

"Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
news:%23ZsqStV6GHA.1256@TK2MSFTNGP04.phx.gbl...
> Your solution looks nice. I'm thinknig this could allow me to run
multiples
> instances of this class simultaneously (to enable object pooling why not).
> However, if I separate this process into a clean class w/o any UI, how can
I
> "reconnect" to the thread that host controls life time ?
> I won't be able to call control.invoke unless I pass the reference of the
> control... but this would mean that the class can only be used in windows
> app...
>
> Steve
>
> "Michael D. Ober" <ober***@.alum.mit.edu.nospam> a écrit dans le message
de
> news: OgtUMpU6GHA.4***@TK2MSFTNGP03.phx.gbl...
> > Instead of using a straight queue, create a new class that inherits from
> > the queue class. Then override the add method to start your background
> > method when a file is added to the queue. You'll need to make the
> > underlying queue thread safe by using synclock ... end synclock
statements
> > (simplest and sufficient) or reader/writer locks. If you only want a
> > single thread doing the actual file processing, you'll also need to
> > include a thread object in the derived class to track the lifetime of
your
> > worker thread.
> >
> > Here's a sample shell to get you started.
> >
> > imports system.io
> > imports system.threading
> >
> > Public Class FileProcessor
> >   inherits system.collections.queue
> >
> >   private th as new thread(addressof FileProcessor)
> >   private so as new syncronizationcontext
> >
> >   private Shadows sub Add(byval File as string)       ' I'm using
Shadows
> > to block all queue.add methods
> >      synclock so
> >         mybase.add file
> >      end synclock
> >
> >      if th.isAlive then exit sub
> >      th.start
> >   end sub
> >
> >   private sub FileProcessor()
> >      do while not queue.isempty
> >          synclock so
> >              dim File as string = queue.dequeue()
> >          end synclock
> >
> >          ' Process your file
> >      loop
> >   end sub
> >
> > end Class
> >
> > Mike Ober
> >
> >
> > "Steve B." <steve_beauge@com.msn_swap_msn_and_com> wrote in message
> > news:OKwKqZT6GHA.4476@TK2MSFTNGP04.phx.gbl...
> >> Hi,
> >>
> >> I'm building an application that performs some long operations against
> >> files.
> >> In order to view the progress, I'm using two background workers.
> >>
> >> The first backgroundworker is listing recursively all files in a folder
> >> and each files are queued into a Queue<FileInfo> object.
> >> I'm not using the Directory.GetFiles("*.*",
SearchOptions.AllDirectories)
> >> because lisiting all files is quite long and I want to start my process
> >> as soon as on file is found.
> >>
> >> The second backgroundworker will execute the process agains each found
> >> files.
> >>
> >> I'm wondering how can I properly code my application in order to run
the
> >> second background worker only when files are filled. Note that I must
run
> >> the background worker from the window thread because the
> >> progressedchanged must change controls properties.
> >>
> >> Since the two background worker are started simultaneously, I need a
way
> >> to block the second BGWorker until the first file has been found. I
using
> >> a while clause but I'm not sure this is a good practice regarding the
> >> performances.
> >>
> >> I'm using by now this code snippet :
> >>
> >> public partial class Form1 : Form
> >> {
> >> /// <summary>
> >> /// Initializes a new instance of the <see cref="T:Form1"/> class.
> >> /// </summary>
> >> public Form1()
> >> {
> >> InitializeComponent();
> >> m_filesToAnalyse = new Queue<FileInfo>();
> >> m_filesToCompare = new Queue<FileScanResult>();
> >> }
> >> private Queue<FileInfo> m_filesToAnalyse;
> >> private bool m_remainingFiles;
> >> private void button1_Click(object sender, EventArgs e)
> >> {
> >> m_remainingFiles = true;
> >> folderBrowsing.RunWorkerAsync(@"I:\Formations");
> >> filesAnalysing.RunWorkerAsync();
> >> }
> >> private void AnalyseFolder(DirectoryInfo dir) // Used for recursive
> >> browsing
> >> {
> >> //Thread.Sleep(50);
> >> folderBrowsing.ReportProgress(0, dir);
> >> Debug.WriteLine("* " + dir.FullName);
> >> foreach (FileInfo file in dir.GetFiles())
> >> {
> >> m_filesToAnalyse.Enqueue(file);
> >> }
> >> foreach (DirectoryInfo subDir in dir.GetDirectories())
> >> {
> >> AnalyseFolder(subDir);
> >> }
> >> }
> >> #region folderBrowsing
> >> private void folderBrowsing_DoWork(object sender, DoWorkEventArgs e)
> >> {
> >> string folder = e.Argument.ToString();
> >> DirectoryInfo dir = new DirectoryInfo(folder);
> >> AnalyseFolder(dir);
> >> }
> >> private void folderBrowsing_ProgressChanged(object sender,
> >> ProgressChangedEventArgs e)
> >> {
> >> listBox1.Items.Insert(0, string.Format(
> >> "Folder : {0}",
> >> ((DirectoryInfo)e.UserState).FullName
> >> ));
> >> }
> >> private void folderBrowsing_RunWorkerCompleted(object sender,
> >> RunWorkerCompletedEventArgs e)
> >> {
> >> m_remainingFiles = false;
> >> listBox1.Items.Insert(0, "parcours fichiers fini");
> >> }
> >> #endregion
> >> #region filesAnalysing
> >> private void filesAnalysing_DoWork(object sender, DoWorkEventArgs e)
> >> {
> >> lock (m_filesToAnalyse)
> >> {
> >> while (m_remainingFiles || m_filesToAnalyse.Count > 0)
> >> {
> >> if(m_filesToAnalyse.Count > 0)
> >> {
> >> FileInfo fi = m_filesToAnalyse.Dequeue();
> >> filesAnalysing.ReportProgress(0, fi);
> >> Debug.WriteLine("- " + fi.FullName);
> >> /*
> >> * Perform here the custom process
> >> */
> >>
> >> );
> >> }
> >> }
> >> }
> >> }
> >> private void filesAnalysing_ProgressChanged(object sender,
> >> ProgressChangedEventArgs e)
> >> {
> >> listBox1.Items.Insert(0, string.Format(
> >> "Fichier : {0}",
> >> ((FileInfo)e.UserState).FullName
> >> ));
> >> }
> >> private void filesAnalysing_RunWorkerCompleted(object sender,
> >> RunWorkerCompletedEventArgs e)
> >> {
> >> listBox1.Items.Insert(0, "analyse fichiers fini");
> >> }
> >> #endregion
> >> }
> >>
> >> Am I on the right way ?
> >>
> >> Thanks in advance,
> >> Steve
> >>
> >
> >
>
>
>

7 Oct 2006 1:41 AM

Brian Gideon

Steve,

It sounds like a blocking queue would be a good fit for the problem. A
blocking queue is similar to a normal queue except that the Dequeue
method is designed to block when the queue is empty. So what you'd do
is have your file searcher thread enqueue files to the queue and the
processing thread will dequeue them when they arrive. Once that thread
finishes processing the file it will call the Dequeue method again.
It's a shame this type of queue isn't included in the framework since
it is so useful. You'd have code own, but be careful. It's
surprisingly difficult to implement correctly.

Brian

Show quote

On Oct 6, 5:58 am, "Steve B." <steve_bea...@com.msn_swap_msn_and_com>
wrote:
> Hi,
>
> I'm building an application that performs some long operations against
> files.
> In order to view the progress, I'm using two background workers.
>
> The first backgroundworker is listing recursively all files in a folder and
> each files are queued into a Queue<FileInfo> object.
> I'm not using the Directory.GetFiles("*.*", SearchOptions.AllDirectories)
> because lisiting all files is quite long and I want to start my process as
> soon as on file is found.
>
> The second backgroundworker will execute the process agains each found
> files.
>
> I'm wondering how can I properly code my application in order to run the
> second background worker only when files are filled. Note that I must run
> the background worker from the window thread because the progressedchanged
> must change controls properties.
>
> Since the two background worker are started simultaneously, I need a way to
> block the second BGWorker until the first file has been found. I using a
> while clause but I'm not sure this is a good practice regarding the
> performances.
>
> [snip]
>
> Thanks in advance,
> Steve