fivesmallq · June 20, 2022 05:43 · Apr 19, 2018 · Apr 19, 2018 · Jan 4, 2017 · Jan 4, 2017
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,9 +30,11 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" or "minWorkerThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  **If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.**  You should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
+ - In ASP.NET, use the ["minIoThreads" or "minWorkerThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  
 
-> **Important Note:** The value specified in this configuration element IS a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
+> **Important Notes:** 
+> * The value specified in this configuration element IS a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
+> * If you are running inside of Azure WebSites, this setting is not exposed through the configuration options and will need to follow the programmatic APIs from within the Application_Start method in Global.asax.
 
  - Outside of ASP.NET, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.  
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,7 +30,7 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.  You should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
+ - In ASP.NET, use the ["minIoThreads" or "minWorkerThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  **If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.**  You should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
 
 > **Important Note:** The value specified in this configuration element IS a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -36,4 +36,4 @@ How to configure this setting:
 
  - Outside of ASP.NET, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.  
 
-> **Important Note:** The value specified in this API call IS NOT a *per-core* setting.  If you want the setting to be 200 at runtime, pass 200 to this API.
+> **Important Note:** The value specified in when calling SetMinThreads IS NOT a *per-core* setting.  If you want the setting to be 200 at runtime, pass 200 to this API regardless of the number of cores the machine has.
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -32,7 +32,8 @@ How to configure this setting:
 
  - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.  You should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
 
-> **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
+> **Important Note:** The value specified in this configuration element IS a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
 
- - Outside of ASP.NET, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
+ - Outside of ASP.NET, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.  
 
+> **Important Note:** The value specified in this API call IS NOT a *per-core* setting.  If you want the setting to be 200 at runtime, pass 200 to this API.
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,7 +30,7 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in web.config.  If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.  However, you should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in machine.config.  If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.  You should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
 
 > **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -20,7 +20,7 @@ If we look at an example error message from StackExchange.Redis (build 1.0.450 o
 	IOCP: (Busy=6,Free=994,Min=4,Max=1000), 
 	WORKER: (Busy=3,Free=997,Min=4,Max=1000)
 
-In the above example, you can see that for IOCP thread there are 6 busy threads and the system is configured to allow 4 minimum threads.  In this case, the client would have likely seen two 500 ms delays because 6 > 4.
+In the above example, you can see that for IOCP thread there are 6 busy threads and the system is configured to allow 4 minimum threads.  In this case, the client would have likely seen two 500 ms delays because 6 > 4.  Note that these ThreadPool usage stats are for the entire client app and are not only threads used by StackExchange.Redis.
 
 Note that StackExchange.Redis can hit timeouts if growth of either IOCP or WORKER threads gets throttled.
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -26,7 +26,7 @@ Note that StackExchange.Redis can hit timeouts if growth of either IOCP or WORKE
 
 Recommendation:
 ---------------
-Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP and WORKER threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 200 or 300, then test and tweak as needed.
+Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP and WORKER threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 100, then test and tweak as needed.
 
 How to configure this setting:
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -5,8 +5,6 @@ The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port"
  - Worker threads are used when for things like processing `Task.Run(…)` or `ThreadPool.QueueUserWorkItem(…)` methods.  These threads are also used by various components in the CLR when work needs to happen on a background thread.
  - IOCP threads are used when asynchronous IO happens (e.g. reading from the network).  
 
-IOCP threads are most interesting from a StackExchange.Redis viewpoint, so we will focus on those here, but the same concepts apply to Worker threads as well.
-
 The thread pool provides new worker threads or I/O completion threads on demand (without any throttling) until it reaches the "Minimum" setting for each type of thread.  By default, the minimum number of threads is set to the number of processors on a system.  
 
 Once the number of existing (busy) threads hits the "minimum" number of threads, the ThreadPool will throttle the rate at which is injects new threads to one thread per 500 milliseconds.  This means that if your system gets a burst of work needing an IOCP thread, it will process that work very quickly.   However, if the burst of work is more than the configured "Minimum" setting, there will be some delay in processing some of the work as the ThreadPool waits for one of two things to happen
@@ -24,6 +22,8 @@ If we look at an example error message from StackExchange.Redis (build 1.0.450 o
 
 In the above example, you can see that for IOCP thread there are 6 busy threads and the system is configured to allow 4 minimum threads.  In this case, the client would have likely seen two 500 ms delays because 6 > 4.
 
+Note that StackExchange.Redis can hit timeouts if growth of either IOCP or WORKER threads gets throttled.
+
 Recommendation:
 ---------------
 Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP and WORKER threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 200 or 300, then test and tweak as needed.

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -26,7 +26,7 @@ In the above example, you can see that for IOCP thread there are 6 busy threads
 
 Recommendation:
 ---------------
-Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 200 or 300, then test and tweak as needed.
+Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP and WORKER threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 200 or 300, then test and tweak as needed.
 
 How to configure this setting:
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,9 +30,9 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in web.config.  
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in web.config.  If you are running inside of Azure WebSites, this setting is not exposed through the configuration options.  However, you should still be able to set this programmatically (see below) from your Application_Start method in global.asax.cs.
 
 > **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
 
- - Outside of ASP.Net, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
+ - Outside of ASP.NET, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
 
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,7 +30,7 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element in web.config.  
 
 > **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -13,7 +13,7 @@ Once the number of existing (busy) threads hits the "minimum" number of threads,
 	1. An existing thread becomes free to process the work
 	2. No existing thread becomes free for 500ms, so a new thread is created.
 
-Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.  Also, it is important to note that when an existing thread stays idle, it will be cleaned up (after 15 seconds of idle time from what I remember).
+Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.  Also, it is important to note that when an existing thread stays idle for longer than 15 seconds (based on what I remember), it will be cleaned up and this cycle of growth and shrinkage can repeat.
 
 If we look at an example error message from StackExchange.Redis (build 1.0.450 or later), you will see that it now prints ThreadPool statistics (see IOCP and WORKER details below).
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -13,7 +13,7 @@ Once the number of existing (busy) threads hits the "minimum" number of threads,
 	1. An existing thread becomes free to process the work
 	2. No existing thread becomes free for 500ms, so a new thread is created.
 
-Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.
+Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.  Also, it is important to note that when an existing thread stays idle, it will be cleaned up (after 15 seconds of idle time from what I remember).
 
 If we look at an example error message from StackExchange.Redis (build 1.0.450 or later), you will see that it now prints ThreadPool statistics (see IOCP and WORKER details below).
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -7,7 +7,7 @@ The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port"
 
 IOCP threads are most interesting from a StackExchange.Redis viewpoint, so we will focus on those here, but the same concepts apply to Worker threads as well.
 
-The thread pool provides new worker threads or I/O completion threads on demand until it reaches the "Minimum" setting for each type of thread.  By default, the minimum number of threads is set to the number of processors on a system.  
+The thread pool provides new worker threads or I/O completion threads on demand (without any throttling) until it reaches the "Minimum" setting for each type of thread.  By default, the minimum number of threads is set to the number of processors on a system.  
 
 Once the number of existing (busy) threads hits the "minimum" number of threads, the ThreadPool will throttle the rate at which is injects new threads to one thread per 500 milliseconds.  This means that if your system gets a burst of work needing an IOCP thread, it will process that work very quickly.   However, if the burst of work is more than the configured "Minimum" setting, there will be some delay in processing some of the work as the ThreadPool waits for one of two things to happen
 	1. An existing thread becomes free to process the work

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,6 +30,9 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.  **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.
+
+> **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
+
  - Outside of ASP.Net, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
 
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,6 +30,6 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.  **Important Note:** that the value specified in this configuration element is a *per-core* setting.  For example, if you want your minIOThreads setting to be 200 on a 4 core machine, you would use `<processModel minIoThreads="50"/>`.
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.  **Important Note:** the value specified in this configuration element is a *per-core* setting.  For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use `<processModel minIoThreads="50"/>`.
  - Outside of ASP.Net, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
 
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -30,6 +30,6 @@ Given the above information, we strongly recommend that customers set the minimu
 
 How to configure this setting:
 
- - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.  **Important Note:** that the value specified in this configuration element is a *per-core* setting.  For example, if you want your minIOThreads setting to be 200 on a 4 core machine, you would use `<processModel minIoThreads="50"/>`.
  - Outside of ASP.Net, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
 
diff --git a/ThreadPool.md b/ThreadPool.md
@@ -5,7 +5,7 @@ The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port"
  - Worker threads are used when for things like processing `Task.Run(…)` or `ThreadPool.QueueUserWorkItem(…)` methods.  These threads are also used by various components in the CLR when work needs to happen on a background thread.
  - IOCP threads are used when asynchronous IO happens (e.g. reading from the network).  
 
-IOCP threads are most interesting from a StackExchange.Redis viewpoint, so we will focus on those.
+IOCP threads are most interesting from a StackExchange.Redis viewpoint, so we will focus on those here, but the same concepts apply to Worker threads as well.
 
 The thread pool provides new worker threads or I/O completion threads on demand until it reaches the "Minimum" setting for each type of thread.  By default, the minimum number of threads is set to the number of processors on a system.  
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -1,4 +1,4 @@
-Some Important Details
+ThreadPool Growth: Some Important Details
 ---------------
 The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port" (aka IOCP) threads.  
 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -15,7 +15,7 @@ Once the number of existing (busy) threads hits the "minimum" number of threads,
 
 Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.
 
-If we look at an example error message from StackExchange.Redis (build 1.0.450 or later), you will see that it now prints ThreadPool statistics.
+If we look at an example error message from StackExchange.Redis (build 1.0.450 or later), you will see that it now prints ThreadPool statistics (see IOCP and WORKER details below).
 
 	System.TimeoutException: Timeout performing GET MyKey, inst: 2, mgr: Inactive, 
 	queue: 6, qu: 0, qs: 6, qc: 0, wr: 0, wq: 0, in: 0, ar: 0, 

diff --git a/ThreadPool.md b/ThreadPool.md
@@ -0,0 +1,35 @@
+Some Important Details
+---------------
+The CLR ThreadPool has two types of threads - "Worker" and "I/O Completion Port" (aka IOCP) threads.  
+
+ - Worker threads are used when for things like processing `Task.Run(…)` or `ThreadPool.QueueUserWorkItem(…)` methods.  These threads are also used by various components in the CLR when work needs to happen on a background thread.
+ - IOCP threads are used when asynchronous IO happens (e.g. reading from the network).  
+
+IOCP threads are most interesting from a StackExchange.Redis viewpoint, so we will focus on those.
+
+The thread pool provides new worker threads or I/O completion threads on demand until it reaches the "Minimum" setting for each type of thread.  By default, the minimum number of threads is set to the number of processors on a system.  
+
+Once the number of existing (busy) threads hits the "minimum" number of threads, the ThreadPool will throttle the rate at which is injects new threads to one thread per 500 milliseconds.  This means that if your system gets a burst of work needing an IOCP thread, it will process that work very quickly.   However, if the burst of work is more than the configured "Minimum" setting, there will be some delay in processing some of the work as the ThreadPool waits for one of two things to happen
+	1. An existing thread becomes free to process the work
+	2. No existing thread becomes free for 500ms, so a new thread is created.
+
+Basically, it means that when the number of Busy threads is greater than Min threads, you are likely paying a 500ms delay before network traffic is processed by the application.
+
+If we look at an example error message from StackExchange.Redis (build 1.0.450 or later), you will see that it now prints ThreadPool statistics.
+
+	System.TimeoutException: Timeout performing GET MyKey, inst: 2, mgr: Inactive, 
+	queue: 6, qu: 0, qs: 6, qc: 0, wr: 0, wq: 0, in: 0, ar: 0, 
+	IOCP: (Busy=6,Free=994,Min=4,Max=1000), 
+	WORKER: (Busy=3,Free=997,Min=4,Max=1000)
+
+In the above example, you can see that for IOCP thread there are 6 busy threads and the system is configured to allow 4 minimum threads.  In this case, the client would have likely seen two 500 ms delays because 6 > 4.
+
+Recommendation:
+---------------
+Given the above information, we strongly recommend that customers set the minimum configuration value for IOCP threads to something larger than the default value.  We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application.  This setting can also impact the performance of other parts of complicated applications, so each customer needs to fine-tune this setting to their specific needs.  A good starting place is 200 or 300, then test and tweak as needed.
+
+How to configure this setting:
+
+ - In ASP.NET, use the ["minIoThreads" configuration setting](https://msdn.microsoft.com/en-us/library/vstudio/7w2sway1(v=vs.100).aspx) under the `<processModel>` configuration element.
+ - Outside of ASP.Net, use the [ThreadPool.SetMinThreads(…)](https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx) API.
+