Non blocking IO is implemented in Windows by a concept called 'IO Completion Ports' (IOCP).
Using IOCP, we can build highly scalable server side applications that can perform asynchronous IO to deliver maximum throughput for large workloads.
Traditionally server side applications were written by assigning one thread to a socket connection. But this approach seriously limited the number of concurrent connections that a server can handle. By using IOCP, we can overcome the "one-thread-per-client" problem, because 'worker' threads are not blocked for IO. Rather there is a separate pool of IO threads called 'Completion Port Threads' that wait on a special kernel level object called 'Completion Port'.
A completion port is a kernel level object that you can bind with a file handle - either a file stream, database connection or a socket stream. Multiple file handles can be bound to a single completion port. The .NET CLR maintains its own completion port and can bind any file handle to it. Each completion port has a queue associate with it. Once a IO operation completes, a message (completion packet) is posted to the queue. IO threads block or 'wait' on this completion port queue, till a message is posted. The waiting IO threads (a.k.a completion port thread) pick up the messages in the queue in FIFO order. Hence any thread may handle any completion message packet. It is important to note that threads are 'woken' in a LIFO order, so chances are that caches are still warm.
The following links throw more light on this:
http://blog.stevensanderson.com/2008/04/05/improve-scalability-in-aspnet-mvc-using-asynchronous-requests/
http://www.codeproject.com/Articles/1052/Developing-a-Truly-Scalable-Winsock-Server-using-I
Why does the .NET Thread Pool have a separate worker thread pool and a Completion Port pool?
I believe that technically there is no fundamental difference in the nature of the threads associated with each pool. Worker threads are meant to do active work, where as Completion Port threads are meant to wait on completion ports. Since IO threads wait on CPs, they may block for longer periods of time. Hence the .NET framework has created separate categories for them. If there was a single pool, then there could be a situation where a high demand on worker threads exhausts all the threads available to dispatch native I/O callbacks,
potentially leading to deadlock.
Looks like in IIS 7, the threading model has undergone drastic changes. More info available here.
Using IOCP, we can build highly scalable server side applications that can perform asynchronous IO to deliver maximum throughput for large workloads.
Traditionally server side applications were written by assigning one thread to a socket connection. But this approach seriously limited the number of concurrent connections that a server can handle. By using IOCP, we can overcome the "one-thread-per-client" problem, because 'worker' threads are not blocked for IO. Rather there is a separate pool of IO threads called 'Completion Port Threads' that wait on a special kernel level object called 'Completion Port'.
A completion port is a kernel level object that you can bind with a file handle - either a file stream, database connection or a socket stream. Multiple file handles can be bound to a single completion port. The .NET CLR maintains its own completion port and can bind any file handle to it. Each completion port has a queue associate with it. Once a IO operation completes, a message (completion packet) is posted to the queue. IO threads block or 'wait' on this completion port queue, till a message is posted. The waiting IO threads (a.k.a completion port thread) pick up the messages in the queue in FIFO order. Hence any thread may handle any completion message packet. It is important to note that threads are 'woken' in a LIFO order, so chances are that caches are still warm.
The following links throw more light on this:
http://blog.stevensanderson.com/2008/04/05/improve-scalability-in-aspnet-mvc-using-asynchronous-requests/
http://www.codeproject.com/Articles/1052/Developing-a-Truly-Scalable-Winsock-Server-using-I
Why does the .NET Thread Pool have a separate worker thread pool and a Completion Port pool?
I believe that technically there is no fundamental difference in the nature of the threads associated with each pool. Worker threads are meant to do active work, where as Completion Port threads are meant to wait on completion ports. Since IO threads wait on CPs, they may block for longer periods of time. Hence the .NET framework has created separate categories for them. If there was a single pool, then there could be a situation where a high demand on worker threads exhausts all the threads available to dispatch native I/O callbacks,
potentially leading to deadlock.
Looks like in IIS 7, the threading model has undergone drastic changes. More info available here.
No comments:
Post a Comment