|
dev
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Size of datatable...in bytes?Is there any way to get or estimate the memory being used by a dataset or a
datatable? I'm writing a caching class and trying to keep track of the size of the data I'm keeping or at least an estimate, to make better decisions on when and what to expire. I realize that without going to unmanaged code, I'm probably only going to be able to get an estimate, but thats ok. Right now I'm doing a rather crude loop over all the columns of a table but I'm not including any overhead for neither columns, rows, tables, or sets. And I'm guessing that they are quite conciderable. Jesper,
I don't think that your datatable or your dataset will be very hugh in size. However all those datarow objects and moreover all those items where they are again referencing too can be of course very huge. By instance a photo item. Just my idea Cor I only have 'primitive' data in my database. Numbers and some strings here
and there. However, the cache is so flexible that it can contain both a single int object, a DataTable, and a full Dataset. And there can easily be 100 datasets and thousands of small int objects in the cache if the user is paging back and forth. So I'd like to put an 'expense' factor on each cache item in the form of (estimated) size divided by how often I expect to be needing it, and then exipre the most 'expensive' items first. Jesper. Hi Jesper,
You might use some memory allocation profiler such as AQTime from AutomatedQA to check out the memory hit. Also you might use WeakReference class for your cache - it will ensure that the memory will be reclaimed (and the cached data lost) when low on memory. -- Show quoteHide quoteMiha Markic [MVP C#] - RightHand .NET consulting & development www.rthand.com SLODUG - Slovene Developer Users Group www.codezone-si.info "Jesper" <no@spam.com> wrote in message news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... > Is there any way to get or estimate the memory being used by a dataset or > a datatable? I'm writing a caching class and trying to keep track of the > size of the data I'm keeping or at least an estimate, to make better > decisions on when and what to expire. > > I realize that without going to unmanaged code, I'm probably only going to > be able to get an estimate, but thats ok. Right now I'm doing a rather > crude loop over all the columns of a table but I'm not including any > overhead for neither columns, rows, tables, or sets. And I'm guessing that > they are quite conciderable. > Miha- I haven't done a lot w/ weakreference but I'm trying to get more
familiar with it. Do you have a quick example or how you'd implement it in this instance? Thanks man! -- Show quoteHide quoteW.G. Ryan MVP (Windows Embedded) TiBA Solutions www.tibasolutions.com | www.devbuzz.com | www.knowdotnet.com "Miha Markic [MVP C#]" <miha at rthand com> wrote in message news:u8Gvb7qOFHA.904@tk2msftngp13.phx.gbl... > Hi Jesper, > > You might use some memory allocation profiler such as AQTime from > AutomatedQA to check out the memory hit. > Also you might use WeakReference class for your cache - it will ensure that > the memory will be reclaimed (and the cached data lost) when low on memory. > > -- > Miha Markic [MVP C#] - RightHand .NET consulting & development > www.rthand.com > SLODUG - Slovene Developer Users Group www.codezone-si.info > > "Jesper" <no@spam.com> wrote in message > news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... > > Is there any way to get or estimate the memory being used by a dataset or > > a datatable? I'm writing a caching class and trying to keep track of the > > size of the data I'm keeping or at least an estimate, to make better > > decisions on when and what to expire. > > > > I realize that without going to unmanaged code, I'm probably only going to > > be able to get an estimate, but thats ok. Right now I'm doing a rather > > crude loop over all the columns of a table but I'm not including any > > overhead for neither columns, rows, tables, or sets. And I'm guessing that > > they are quite conciderable. > > > > Hi Bill,
I don't have any example handy, however it should be prety straightforward. Instead of holding reference to, i.e., dataset, you hold reference to a WeakReference instance which in turn holds reference to DataSet instance (its property Target). If WeakReference.Target returns you null you know that the DataSet instance is gone. -- Show quoteHide quoteMiha Markic [MVP C#] - RightHand .NET consulting & development www.rthand.com SLODUG - Slovene Developer Users Group www.codezone-si.info "W.G. Ryan eMVP" <WilliamRyan@NoSpam.gmail.com> wrote in message news:eKJdzLyOFHA.3336@TK2MSFTNGP09.phx.gbl... > Miha- I haven't done a lot w/ weakreference but I'm trying to get more > familiar with it. Do you have a quick example or how you'd implement it in > this instance? > > Thanks man! > > -- > W.G. Ryan MVP (Windows Embedded) > > TiBA Solutions > www.tibasolutions.com | www.devbuzz.com | www.knowdotnet.com > "Miha Markic [MVP C#]" <miha at rthand com> wrote in message > news:u8Gvb7qOFHA.904@tk2msftngp13.phx.gbl... >> Hi Jesper, >> >> You might use some memory allocation profiler such as AQTime from >> AutomatedQA to check out the memory hit. >> Also you might use WeakReference class for your cache - it will ensure > that >> the memory will be reclaimed (and the cached data lost) when low on > memory. >> >> -- >> Miha Markic [MVP C#] - RightHand .NET consulting & development >> www.rthand.com >> SLODUG - Slovene Developer Users Group www.codezone-si.info >> >> "Jesper" <no@spam.com> wrote in message >> news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... >> > Is there any way to get or estimate the memory being used by a dataset > or >> > a datatable? I'm writing a caching class and trying to keep track of >> > the >> > size of the data I'm keeping or at least an estimate, to make better >> > decisions on when and what to expire. >> > >> > I realize that without going to unmanaged code, I'm probably only going > to >> > be able to get an estimate, but thats ok. Right now I'm doing a rather >> > crude loop over all the columns of a table but I'm not including any >> > overhead for neither columns, rows, tables, or sets. And I'm guessing > that >> > they are quite conciderable. >> > >> >> > > Jesper - you can serialize it to a memorystream and do a memorystrem.length.
That's a good enough estimate. Not saying that it'd be very efficient :) - Sahil Malik [MVP] http://codebetter.com/blogs/sahil.malik/ Show quoteHide quote "Jesper" <no@spam.com> wrote in message news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... > Is there any way to get or estimate the memory being used by a dataset or > a datatable? I'm writing a caching class and trying to keep track of the > size of the data I'm keeping or at least an estimate, to make better > decisions on when and what to expire. > > I realize that without going to unmanaged code, I'm probably only going to > be able to get an estimate, but thats ok. Right now I'm doing a rather > crude loop over all the columns of a table but I'm not including any > overhead for neither columns, rows, tables, or sets. And I'm guessing that > they are quite conciderable. > Thats a pretty good idea. I think I'll do that in my testing environment and
see if I can catch an approximate pattern to the size, and then replace it with more efficient code. Show quoteHide quote "Sahil Malik [MVP]" <contactmethrumyblog@nospam.com> wrote in message news:u$r2C9qOFHA.1096@tk2msftngp13.phx.gbl... > Jesper - you can serialize it to a memorystream and do a > memorystrem.length. That's a good enough estimate. > Not saying that it'd be very efficient :) > > - Sahil Malik [MVP] > http://codebetter.com/blogs/sahil.malik/ > > > "Jesper" <no@spam.com> wrote in message > news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... >> Is there any way to get or estimate the memory being used by a dataset or >> a datatable? I'm writing a caching class and trying to keep track of the >> size of the data I'm keeping or at least an estimate, to make better >> decisions on when and what to expire. >> >> I realize that without going to unmanaged code, I'm probably only going >> to be able to get an estimate, but thats ok. Right now I'm doing a rather >> crude loop over all the columns of a table but I'm not including any >> overhead for neither columns, rows, tables, or sets. And I'm guessing >> that they are quite conciderable. >> > > Hi Sahil,
Just a note that serialized size might be less than actualy one since not all data might be serialized. And, ah, DataSet serializes into an XML file btw. It will give him way greater memory size than actual is. Perhaps both issues combined might give him exact size :-) -- Show quoteHide quoteMiha Markic [MVP C#] - RightHand .NET consulting & development www.rthand.com SLODUG - Slovene Developer Users Group www.codezone-si.info "Sahil Malik [MVP]" <contactmethrumyblog@nospam.com> wrote in message news:u$r2C9qOFHA.1096@tk2msftngp13.phx.gbl... > Jesper - you can serialize it to a memorystream and do a > memorystrem.length. That's a good enough estimate. > Not saying that it'd be very efficient :) > > - Sahil Malik [MVP] > http://codebetter.com/blogs/sahil.malik/ > > > "Jesper" <no@spam.com> wrote in message > news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... >> Is there any way to get or estimate the memory being used by a dataset or >> a datatable? I'm writing a caching class and trying to keep track of the >> size of the data I'm keeping or at least an estimate, to make better >> decisions on when and what to expire. >> >> I realize that without going to unmanaged code, I'm probably only going >> to be able to get an estimate, but thats ok. Right now I'm doing a rather >> crude loop over all the columns of a table but I'm not including any >> overhead for neither columns, rows, tables, or sets. And I'm guessing >> that they are quite conciderable. >> > > Miha you are right. To get around that, just use DataSetSurrogate (search
the MS KB for that). Anyway, it will always be "proportionate" :). So it's a good design time measure. - Sahil Malik [MVP] http://codebetter.com/blogs/sahil.malik/ Show quoteHide quote "Miha Markic [MVP C#]" <miha at rthand com> wrote in message news:O2UujgrOFHA.2736@TK2MSFTNGP09.phx.gbl... > Hi Sahil, > > Just a note that serialized size might be less than actualy one since not > all data might be serialized. > And, ah, DataSet serializes into an XML file btw. It will give him way > greater memory size than actual is. > Perhaps both issues combined might give him exact size :-) > > -- > Miha Markic [MVP C#] - RightHand .NET consulting & development > www.rthand.com > SLODUG - Slovene Developer Users Group www.codezone-si.info > > "Sahil Malik [MVP]" <contactmethrumyblog@nospam.com> wrote in message > news:u$r2C9qOFHA.1096@tk2msftngp13.phx.gbl... > > Jesper - you can serialize it to a memorystream and do a > > memorystrem.length. That's a good enough estimate. > > Not saying that it'd be very efficient :) > > > > - Sahil Malik [MVP] > > http://codebetter.com/blogs/sahil.malik/ > > > > > > "Jesper" <no@spam.com> wrote in message > > news:uKWflaqOFHA.3296@TK2MSFTNGP15.phx.gbl... > >> Is there any way to get or estimate the memory being used by a dataset or > >> a datatable? I'm writing a caching class and trying to keep track of the > >> size of the data I'm keeping or at least an estimate, to make better > >> decisions on when and what to expire. > >> > >> I realize that without going to unmanaged code, I'm probably only going > >> to be able to get an estimate, but thats ok. Right now I'm doing a rather > >> crude loop over all the columns of a table but I'm not including any > >> overhead for neither columns, rows, tables, or sets. And I'm guessing > >> that they are quite conciderable. > >> > > > > > >  
Other interesting topics
SQL Server does not exist
Join two datatables populated from different sources? memory leak in SqlDataAdapter.Fill method? ADO.Net Connection Pooling Problem with Oracle DataView.RowFilter issue Formatting a SQL query ConnectionTimeout is always 15 coming back from the DataLinksClass dialog Sending XML From .Net To SQLServer ANN: Microsoft webcast on DataSets DataReader Vs DataSet |
|||||||||||||||||||||||
