|
dev
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Using CSharpCodeProvider to create MANY assembliesfunctions in my app. In my current implementation, All of these functions are generated in one swipe... thus they all invoke the compiler once, and create only one DLL. My new requirement tells me that I need to be more dynamic... that I can't queue up all of these functions... I actually need to do them more on demand. The number of times this happens can be as many as 10,000 times... each with a different calculation. This would mean that I invoke the compiler 10,000 times and create 10,000 in-memory DLLs. This idea scares me... can .NET handle that many DLLs being linked to the app? Does anyone know about the performance overhead of such an endeavor? I am about to write some tests to gather data, but I was wondering if anyone out there has experience with this type of execution. Am I going to run into performance penalties for invoking the compiler that many times? Am I going to run into performance penalties for having that many DLLs in the app domain? If so, are there any workarounds that anyone knows of? Thanks, Brian Brian,
I would say you are probably going to start seeing some issues when you try and load 10,000 assemblies into an app-domain. The assemblies themselves are small, but the CLR is still going to have overhead in the form of assembly, module, and type info for each assembly. As for your requirement, does it state that you have to have separate assemblies, or that you have to be able to call them dynamically (and not queue it all up). If the case is the former, then I would say revisit the requirements. What's the justification for generating all those assemblies? If the case is the former, I would look into creating an in-memory assembly, with a single module, and add new global methods/types as needed and implementing them using IL, instead of the codedom provider and compiling separately. -- Show quote- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com "Bilz" <BrianGeni***@gmail.com> wrote in message news:9c89a7d6-08e8-4a8e-a839-d6e53a3a0bb8@y5g2000hsf.googlegroups.com... >I am planning to use the CSharpCodeProvider to generate some compiled > functions in my app. > > In my current implementation, All of these functions are generated in > one swipe... thus they all invoke the compiler once, and create only > one DLL. My new requirement tells me that I need to be more > dynamic... that I can't queue up all of these functions... I actually > need to do them more on demand. > > The number of times this happens can be as many as 10,000 times... > each with a different calculation. This would mean that I invoke the > compiler 10,000 times and create 10,000 in-memory DLLs. This idea > scares me... can .NET handle that many DLLs being linked to the app? > > Does anyone know about the performance overhead of such an endeavor? > I am about to write some tests to gather data, but I was wondering if > anyone out there has experience with this type of execution. Am I > going to run into performance penalties for invoking the compiler that > many times? Am I going to run into performance penalties for having > that many DLLs in the app domain? If so, are there any workarounds > that anyone knows of? > > Thanks, > Brian Sorry, the second "former" should be "latter".
-- Show quote- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote in message news:%23Mwkj5FMIHA.3976@TK2MSFTNGP03.phx.gbl... > Brian, > > I would say you are probably going to start seeing some issues when you > try and load 10,000 assemblies into an app-domain. The assemblies > themselves are small, but the CLR is still going to have overhead in the > form of assembly, module, and type info for each assembly. > > As for your requirement, does it state that you have to have separate > assemblies, or that you have to be able to call them dynamically (and not > queue it all up). > > If the case is the former, then I would say revisit the requirements. > What's the justification for generating all those assemblies? > > If the case is the former, I would look into creating an in-memory > assembly, with a single module, and add new global methods/types as needed > and implementing them using IL, instead of the codedom provider and > compiling separately. > > > -- > - Nicholas Paldino [.NET/C# MVP] > - mvp@spam.guard.caspershouse.com > > "Bilz" <BrianGeni***@gmail.com> wrote in message > news:9c89a7d6-08e8-4a8e-a839-d6e53a3a0bb8@y5g2000hsf.googlegroups.com... >>I am planning to use the CSharpCodeProvider to generate some compiled >> functions in my app. >> >> In my current implementation, All of these functions are generated in >> one swipe... thus they all invoke the compiler once, and create only >> one DLL. My new requirement tells me that I need to be more >> dynamic... that I can't queue up all of these functions... I actually >> need to do them more on demand. >> >> The number of times this happens can be as many as 10,000 times... >> each with a different calculation. This would mean that I invoke the >> compiler 10,000 times and create 10,000 in-memory DLLs. This idea >> scares me... can .NET handle that many DLLs being linked to the app? >> >> Does anyone know about the performance overhead of such an endeavor? >> I am about to write some tests to gather data, but I was wondering if >> anyone out there has experience with this type of execution. Am I >> going to run into performance penalties for invoking the compiler that >> many times? Am I going to run into performance penalties for having >> that many DLLs in the app domain? If so, are there any workarounds >> that anyone knows of? >> >> Thanks, >> Brian > > On Nov 26, 2:03 pm, "Nicholas Paldino [.NET/C# MVP]"
<m...@spam.guard.caspershouse.com> wrote: > Brian, No, there is no requirement to create multiple DLLs... just that the> > I would say you are probably going to start seeing some issues when you > try and load 10,000 assemblies into an app-domain. The assemblies > themselves are small, but the CLR is still going to have overhead in the > form of assembly, module, and type info for each assembly. > functions be dynamically generated. The functions look to the user like math equations... wrap their math equations with C# methods, compile them, and run them against their inputs. I will look into the idea of inserting methods via IL. I have never done anything like that. If I can't use C# code, it might defeat the purpose, since I am using this method to avoid parsing their functions. They write something like ((input1 * input2)/input3) and I write that out directly as a C# method with 3 parameters. Thanks, B Brian,
Well, you would have to parse it anyways to get the parameter names. Beyond that, you would have to map the operators and whatnot to IL (which might be no small task). You might be able to get away with compiling to memory and then taking the IL from the compiled unit and then placing it dynamically in your type. I would test creating separate assemblies, and see if you see any noticable change in your app for the load you would expect to normally handle. If it works, then I would leave it at that. Only if it doesn't would I venture down this path. -- Show quote- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com "Bilz" <BrianGeni***@gmail.com> wrote in message news:f977ead5-3344-4eca-bac5-1182068dd23c@e25g2000prg.googlegroups.com... > On Nov 26, 2:03 pm, "Nicholas Paldino [.NET/C# MVP]" > <m...@spam.guard.caspershouse.com> wrote: >> Brian, >> >> I would say you are probably going to start seeing some issues when >> you >> try and load 10,000 assemblies into an app-domain. The assemblies >> themselves are small, but the CLR is still going to have overhead in the >> form of assembly, module, and type info for each assembly. >> > > No, there is no requirement to create multiple DLLs... just that the > functions be dynamically generated. The functions look to the user > like math equations... wrap their math equations with C# methods, > compile them, and run them against their inputs. > > I will look into the idea of inserting methods via IL. I have never > done anything like that. If I can't use C# code, it might defeat the > purpose, since I am using this method to avoid parsing their > functions. They write something like ((input1 * input2)/input3) and I > write that out directly as a C# method with 3 parameters. > > Thanks, > B On Nov 26, 4:05 pm, "Nicholas Paldino [.NET/C# MVP]"
<m...@spam.guard.caspershouse.com> wrote: > I would test creating separate assemblies, and see if you see any Yes, I agree with that... I am really just investigating right now. I> noticable change in your app for the load you would expect to normally > handle. If it works, then I would leave it at that. Only if it doesn't > would I venture down this path. just don't want to go down a design path that falls apart as it scales. I looked into the DynamicMethod class... I thought about what you said... compile it into a new assembly and transfer the IL over to the primary assembly. Is there a way to unload the temp assembly? I don't know that I have ever UNloaded an assembly dynamically. Thanks for ALL the help, B Bilz,
If you compile the assembly, I don't know that it gets loaded by the CLR. There might be a managed representation of it in memory, but I don't know that it's considered loaded. If it is loaded into memory though, you are going to have to do it in a separate app domain, so that you can unload it. Is the parsing that difficult to do? I ask, because if it is not, then I would recommend building some sort of parser/graph tree for your expressions and then creating the IL from that. If you are using .NET 3.5/C# 3.0, you could even use Expressions to help build your methods for you, as it should make it much, much easier. -- Show quote- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com "Bilz" <BrianGeni***@gmail.com> wrote in message news:a65699ab-f6cf-432b-be57-04b552667866@d4g2000prg.googlegroups.com... > On Nov 26, 4:05 pm, "Nicholas Paldino [.NET/C# MVP]" > <m...@spam.guard.caspershouse.com> wrote: >> I would test creating separate assemblies, and see if you see any >> noticable change in your app for the load you would expect to normally >> handle. If it works, then I would leave it at that. Only if it doesn't >> would I venture down this path. > > Yes, I agree with that... I am really just investigating right now. I > just don't want to go down a design path that falls apart as it > scales. > > I looked into the DynamicMethod class... I thought about what you > said... compile it into a new assembly and transfer the IL over to the > primary assembly. Is there a way to unload the temp assembly? I > don't know that I have ever UNloaded an assembly dynamically. > > Thanks for ALL the help, > B Bilz <BrianGeni***@gmail.com> writes:
> I looked into the DynamicMethod class... I thought about what you Assemblies are only unloaded, if the (last) AppDomain using it is> said... compile it into a new assembly and transfer the IL over to the > primary assembly. Is there a way to unload the temp assembly? I > don't know that I have ever UNloaded an assembly dynamically. unloaded. So temporary assemblies should be loaded only in an own AppDomain to allow later unloading. Best regards, Martin  |
|||||||||||||||||||||||
Other interesting topics