Home All Groups Group Topic Archive Search About

Building a Scalable (100K+ User) Socket Server in .Net

Author
12 Jul 2006 5:52 AM
Chris Mullins
One of the things I've spent the last several years working on is a highly
scalable socket server written in C#.

The SoapBox Server has recently been tested to well over 100k simultanous
users. This means it's handling 100k TCP connections on a Windows Platform
without any trouble at all.

I finally got around to writing a blog post that describes the different
architectures we've gone through, and the pros and cons of each. All of
these architectures scale fairly well, but there have been some surprising
results. These architectures have included using the system threadpool,
managing our own thread pool, doing work on the IOCP threads, and a few
other things thrown in for good luck.

I've seen so many people asking for guidance in this area, and I hope this
serves people as a good starting point.

http://makeashorterlink.com/?R2EE5246D

All feedback is welcome, either here or on the blog...

--
Chris Mullins
Coversant, Inc.

Author
13 Jul 2006 12:36 PM
Nick Malik [Microsoft]
Hi Chris,

Just so you know, the short link to your blog entry is below.  You don't
have to use a link shortener to allow folks to find the entry.

http://www.coversant.net/dotnetnuke/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx

After you make the post to the blog, go back to your blog home page.  You'll
see the list of your recent posts with a summary or the first few sentences
under each.  Simply grab the link from there.

--
--- Nick Malik [Microsoft]
    MCSD, CFPS, Certified Scrummaster
    http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
   I do not answer questions on behalf of my employer.  I'm just a
programmer helping programmers.
--
Show quote
"Chris Mullins" <cmull***@yahoo.com> wrote in message
news:%23rrCqdXpGHA.4268@TK2MSFTNGP04.phx.gbl...
> One of the things I've spent the last several years working on is a highly
> scalable socket server written in C#.
>
> The SoapBox Server has recently been tested to well over 100k simultanous
> users. This means it's handling 100k TCP connections on a Windows Platform
> without any trouble at all.
>
> I finally got around to writing a blog post that describes the different
> architectures we've gone through, and the pros and cons of each. All of
> these architectures scale fairly well, but there have been some surprising
> results. These architectures have included using the system threadpool,
> managing our own thread pool, doing work on the IOCP threads, and a few
> other things thrown in for good luck.
>
> I've seen so many people asking for guidance in this area, and I hope this
> serves people as a good starting point.
>
> http://makeashorterlink.com/?R2EE5246D
>
> All feedback is welcome, either here or on the blog...
>
> --
> Chris Mullins
> Coversant, Inc.
>
>
Author
13 Jul 2006 5:28 PM
Chris Mullins
"Nick Malik [Microsoft]" <nickmalik@hotmail.nospam.com> wrote
> Hi Chris,
>
> Just so you know, the short link to your blog entry is below.  You don't
> have to use a link shortener to allow folks to find the entry.
> http://www.coversant.net/dotnetnuke/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx


Well, to be fair, the link I made shorter, and the link you posted aren't
the same.

I wanted to see how well Google Analytics worked, so I annoted the link to
indicate the clicker came from a UseNet post that I made. Doing this made
the link a bit longer than I would normally like to see (it split across a
line), so I ran it through a link shortener.

The actual link I used was:
http://www.coversant.net/dotnetnuke/Coversant/Blogs/tabid/88/EntryID/10/Default.aspx?utm_source=UseNet%2BPost&utm_medium=UseNet&utm_term=Short&utm_content=Technical%2Bstuff%2Bto%2BUseNet&utm_campaign=UseNet

Seeing how the analytics works is pretty interesting....

> After you make the post to the blog, go back to your blog home page.
> You'll see the list of your recent posts with a summary or the first few
> sentences under each.  Simply grab the link from there.

Ummmmm. Thanks. :)

--
Chris Mullins
http://www.coversant.net/blogs/cmullins
(IOCP and Threading Expert, Blogging Novice)
Author
13 Jul 2006 2:00 PM
Carl Daniel [VC++ MVP]
Chris Mullins wrote:
Show quote
> One of the things I've spent the last several years working on is a
> highly scalable socket server written in C#.
>
> The SoapBox Server has recently been tested to well over 100k
> simultanous users. This means it's handling 100k TCP connections on a
> Windows Platform without any trouble at all.
>
> I finally got around to writing a blog post that describes the
> different architectures we've gone through, and the pros and cons of
> each. All of these architectures scale fairly well, but there have
> been some surprising results. These architectures have included using
> the system threadpool, managing our own thread pool, doing work on
> the IOCP threads, and a few other things thrown in for good luck.
>
> I've seen so many people asking for guidance in this area, and I hope
> this serves people as a good starting point.
>
> http://makeashorterlink.com/?R2EE5246D
>
> All feedback is welcome, either here or on the blog...

Thanks for sharing - great article!

-cd
Author
13 Jul 2006 5:33 PM
Chris Mullins
"Carl Daniel [VC++ MVP]" wrote

>>  [Writing a Scalable Socket Server]
>> http://makeashorterlink.com/?R2EE5246D

> Thanks for sharing - great article!

Any suggestions for how to improve things? The great thing about UseNet is
the wealth of knowledge out here, people wise. Getting hold of that
knowledge can be tough, but please, if you have suggestions fire away!

Author
13 Jul 2006 6:14 PM
Carl Daniel [VC++ MVP]
"Chris Mullins" <cmull***@yahoo.com> wrote in message
news:OgN1pJqpGHA.4188@TK2MSFTNGP03.phx.gbl...
> "Carl Daniel [VC++ MVP]" wrote
>
>>>  [Writing a Scalable Socket Server]
>>> http://makeashorterlink.com/?R2EE5246D
>
>> Thanks for sharing - great article!
>
> Any suggestions for how to improve things? The great thing about UseNet is
> the wealth of knowledge out here, people wise. Getting hold of that
> knowledge can be tough, but please, if you have suggestions fire away!

I think you're to the (really) hard part now - do as SQL Server does (as
discussed in a previous thread).   You need to keep your threads from being
blocked and avoid context switches like the plague.  Unfortunately, ADO.NET
doesn't help you out a lot there.  It'd be interesting to see (as you've
already commented) how much more you'd get out at the high end just by using
lockfree queues on high-end systems while sticking with the locking queue on
low(er) end systems.

-cd
Author
13 Jul 2006 6:40 PM
Chris Mullins
"Carl Daniel [VC++ MVP]" wrote:
> It'd be interesting to see (as you've already commented) how much more
> you'd get out at the high end just by using lockfree queues on high-end
> systems while sticking with the locking queue on low(er) end systems.

My next topic will probably be just that. I wrote a mean little program that
is contention bound, and uses all the different types of locks. I ran this
on single x86, dualx86, single x64, dual x64, dual x64 with hyperthreading,
dual itanium2, quad itanium2, and 16 processor itanium2.

The resulting graphs are.... interesting.

Now I just need to find time to write that up...

Author
13 Jul 2006 6:46 PM
Jon Skeet [C# MVP]
Chris Mullins <cmull***@yahoo.com> wrote:
> "Carl Daniel [VC++ MVP]" wrote:
> > It'd be interesting to see (as you've already commented) how much more
> > you'd get out at the high end just by using lockfree queues on high-end
> > systems while sticking with the locking queue on low(er) end systems.
>
> My next topic will probably be just that. I wrote a mean little program that
> is contention bound, and uses all the different types of locks. I ran this
> on single x86, dualx86, single x64, dual x64, dual x64 with hyperthreading,
> dual itanium2, quad itanium2, and 16 processor itanium2.
>
> The resulting graphs are.... interesting.
>
> Now I just need to find time to write that up...

If you have the time to run the tests with my "feature-rich" locks, I'd
be really interested in seeing the results. See
http://www.pobox.com/~skeet/csharp/miscutil/usage/locking.html for
usage, download link etc.

It sounds like they wouldn't be appropriate for your situation, but it
would be good to get more diverse stats on the penalty for using my
code over the "plain" locking.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet   Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Author
13 Jul 2006 7:07 PM
Chris Mullins
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote
>
> If you have the time to run the tests with my "feature-rich" locks, I'd
> be really interested in seeing the results. See
> http://www.pobox.com/~skeet/csharp/miscutil/usage/locking.html for
> usage, download link etc.
>

I recently (just this week) took a long and deep look through Jeff Richters
"Power Threading" library. It's online at:
http://www.wintellect.com/Resources.aspx
(signing up for an account is free, then you can download the source).

As somebody who takes quite a bit of pride in my knowledge of multi-threaded
code, I have to say I was humbled.

I learned more about locking in 4 hours this past tuesday evening than I've
leared in a long, long time - possibly ever. Some of his source code
commends were... eye opening.
// NOTE: Just reading here (as compared to repeatedly calling Exchange)
// improves performance because writing forces all CPUs to update this value

// Don't do variable = value because reordered by compiler and/or CPU

There was stuff like this throughout - as well as stuff that I wish I could
wrap my head arond.

I quite liked his abstract ResourceLock implementation, and the concrete
variations on it.

Unfortunatly I can't use this library, as the licensing of it prevents it
from being run with Mono. I'll have to send him an email and see if I can
get a relaxed license version.

--
Chris Mullins
Author
14 Jul 2006 4:37 AM
Carl Daniel [VC++ MVP]
Chris Mullins wrote:
Show quote
> "Jon Skeet [C# MVP]" <sk***@pobox.com> wrote
>>
>> If you have the time to run the tests with my "feature-rich" locks,
>> I'd be really interested in seeing the results. See
>> http://www.pobox.com/~skeet/csharp/miscutil/usage/locking.html for
>> usage, download link etc.
>>
>
> I recently (just this week) took a long and deep look through Jeff
> Richters "Power Threading" library. It's online at:
> http://www.wintellect.com/Resources.aspx
> (signing up for an account is free, then you can download the source).
>
> As somebody who takes quite a bit of pride in my knowledge of
> multi-threaded code, I have to say I was humbled.
>
> I learned more about locking in 4 hours this past tuesday evening
> than I've leared in a long, long time - possibly ever. Some of his
> source code commends were... eye opening.
> // NOTE: Just reading here (as compared to repeatedly calling
> Exchange) // improves performance because writing forces all CPUs to
> update this value
> // Don't do variable = value because reordered by compiler and/or CPU
>
> There was stuff like this throughout - as well as stuff that I wish I
> could wrap my head arond.
>
> I quite liked his abstract ResourceLock implementation, and the
> concrete variations on it.
>
> Unfortunatly I can't use this library, as the licensing of it
> prevents it from being run with Mono. I'll have to send him an email
> and see if I can get a relaxed license version.

Cool library - definitely some good stuff in there.  Kinda short on
documentation though :)  Makes for more interesting reading that way, I
guess.

-cd
Author
14 Jul 2006 6:07 AM
Barry Kelly
"Chris Mullins" <cmull***@yahoo.com> wrote:

> I recently (just this week) took a long and deep look through Jeff Richters
> "Power Threading" library. It's online at:
> http://www.wintellect.com/Resources.aspx
> (signing up for an account is free, then you can download the source).
>
> As somebody who takes quite a bit of pride in my knowledge of multi-threaded
> code, I have to say I was humbled.

Yes, memory models are mind-bending - especially when you bring compiler
optimizations into mind as well. Are we really certain that the JIT
compiler respects Thread.MemoryBarrier(), or does it just affect the
CPU? :) It's certainly enough to scare you away from playing with
sophisticated lock-free programming unless you've got a lot of time and
hardware to play with to prove and test correctness.

-- Barry


AddThis Social Bookmark Button