我有HTTP客户端，它基本上是调用针对HTTP服务器的多个Web请求。而且我在执行一个线程池线程（同步调用）每个HTTP请求，而默认使用30 TCP（使用httpwebrequest.servicepoint - 的http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.servicepoint.aspx ）。而基于该系统，我管理上，可以有〜一千分之五百等待I / O（HTTP响应）线程池中的线程

I have HTTP client which basically invokes multiple web requests against HTTP server. And I execute each HTTP request in a thread pool thread (synchronous call), and by default uses 30 TCP (using httpwebrequest.servicepoint - http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.servicepoint.aspx ). And based on the system i am managing, there can be ~500/1000 thread pool threads waiting for I/O (http response)

现在，我想知道的我需要限制我使用，以及线程数？ （为前， http://msdn.microsoft .COM / EN-US /库/ ee789351（V = vs.110）的.aspx System.Threading.Tasks - 限制并发任务的数量的）

Now, I am wondering do I need to limit the number of threads I use as well? (for ex, http://msdn.microsoft.com/en-us/library/ee789351(v=vs.110).aspx System.Threading.Tasks - Limit the number of concurrent Tasks )

修改

是的，我想我需要，限制我因为尽管这些线程都在它们占用的资源等待状态使用的线程数。这样我可以控制我用了，这使得它更容易为我的组件与其他人相结合，而不会引起他们的饥饿/争用资源/线程资源/线程数。

Yes, I think I need to, limit the number of threads I use as even though these threads are in wait state they take up resources. This way I can control number of resources/threads I use up which makes it easier for my component to be integrated with others without causing them for starvation/contention for resource/threads.

编辑2

我已经决定彻底​​拥抱异步模式，使我不会使用线程池中的线程来执行HTTP请求，而我可以简单地靠操作系统内核和我的合作/ O完成端口的线程（S），这将确保建成后的反应会在回调中发送（这样我可以充分利用的CPU以及资源）。我目前想使用（webclient.uploaddatataskasync）http://msdn.microsoft.com/en-us/library/system.net.webclient.uploaddatataskasync(v=vs.110).aspx,并相应地更新code。 （夫妇详情参考：HttpWebRequest和I / O完成端口的，How没有.NET使用IO线程或IO完成端口的？）

I have decided to completely embrace async model so that i won't be using thread pool threads to execute http requests, rather I can simply rely on "collaboration of OS Kernel and I/O completion port thread(s)" which will ensure that upon completion response will be sent in a callback (this way i can best use of cpu as well as resource). I am currently thinking of using (webclient.uploaddatataskasync) http://msdn.microsoft.com/en-us/library/system.net.webclient.uploaddatataskasync(v=vs.110).aspx, and update the code accordingly. (couple of references for details: HttpWebRequest and I/O completion ports, How does .NET make use of IO Threads or IO Completion Ports? )

修改3

基本上我曾用异步网络I / O .NET上述的API基本上打消了我的并行库的使用。有关详细信息，请参阅下面的答案（我已经将其添加为方便起见，以防万一，如果有人有兴趣！）。

Basically i have used "async network I/O .net APIs as mentioned above" which essentially removed usage of my parallel library. For details, please see the below answer (i have added it for convenience, just in case if anyone is interested!).

伪code给一​​个想法使用Web客户端我如何调用Web请求的

psuedo code to give an idea how I am invoking web requests using webclient

//psudeo code to represents there can be varibale number of requests
//these can be ~500 to ~1000
foreach(var request in requests)
{
    //psudeo code which basically executes webrequest in threadpool thread
    //MY QUESTION: Is it OK to create as many worker threads as number rrequests
    //and simply let them wait on a semaphore, on should i limit the concurrency?
    MyThreadPoolConcurrentLibrary.ExedcuteAction(() =>
        {            
            var sem = new Semaphore(initialCount: 50, maximumCount: 50.Value);
            try
            {
                //using semaphore as the HTTP Server which i am taking to recommend 
                //to send '50' parallel requests in '30' TCP Connections
                sem.WaitOne();
                //using my custom webclient, so that i can configure 'tcp' connections 
                //(servicepoint connection limit) and ssl validation etc.
                using (MyCustomWebClient client = new MyCustomWebClient())
                {
                    //http://msdn.microsoft.com/en-us/library/tdbbwh0a(v=vs.110).aspx
                    //basically the worker thread simply waits here
                    client.UploadData(address: "urladdress", data: bytesdata);
                }
            }
            finally
            {
                sem.Release(1);
            }
        });
}
MyThreadPoolConcurrentLibrary.WaitAll(/*...*/);

基本上，我应该做些什么来限制线程我消耗的数量，还是让线程池照顾它（即情况下，如果我的应用程序达到线程池的最大线程限制，任何方式将请求排队 - 这样我就可以单纯地依靠它）

Basically should I do something to limit the number of threads I consume, or let the thread pool take care of it (i.e. in case if my app reaches thread pool's maximum thread limit, it any way queues the request - so I can simply rely on it)

*伪code这应该显示我的自定义Web客户端，我配置的TCP连接，SSL验证等。

class MyCustomWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(address);            
        request.KeepAlive = true;
        request.Timeout = 300;
        request.ServicePoint.ConnectionLimit = TCPConnectionsLimit;
        request.ServerCertificateValidationCallback = this.ServerCertificateValidationCallback;
        return request;

    }
    private bool ServerCertificateValidationCallback(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certificate, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
    {
        throw new NotImplementedException();
    }
}

最好的问候。

Best Regards.

推荐答案

由于我进行网络I / O（HTTP Web请求），这是不好的主意使用'同步'httpwebrequests，让线程池中的线程来阻止同步调用。所以，我用异步网络I / O操作（Web客户端的异步工作方式），如问题按照自评的建议如上所述。它会自动删除线程在我的组件数的用法 - 详情请参阅psudeo code以下片段...

Since I am performing network I/O (http web requests), it is not good idea use 'synchronous' httpwebrequests and let the thread pool threads to block in sync calls. So, i have used 'async network i/o operations (web client's async task methods) as mentioned above in the question as per the suggestions from comments. It automatically removed usage of number of threads in my component - for details, please see below psudeo code snippet...

下面是帮助我很容易适应一些C＃5.0异步概念一些有用的链接（异步/等待）：

深潜视频（异步的很好的解释/待机状态机）的http://channel9.msdn.com/events/TechDays/Techdays-2014-the-Netherlands/Async-programming-deep-dive

Deep Dive Video (good explanation of async/await state machine) http://channel9.msdn.com/events/TechDays/Techdays-2014-the-Netherlands/Async-programming-deep-dive

http://blog.stephencleary.com/2013/ 11 /那里 - 是 - 没有thread.html

异步/计谋错误处理：的的 http://www.interact-sw.co.uk/iangblog/2010/11/01/csharp5-async-exceptions ，的 http://msdn.microsoft.com/en-us/library/0yd65esw.aspx ，How为更好地了解code /从报表＆QUOT;异步 - 处理多个异常＆QUOT;文章？

尼斯书：的的http://www.amazon.com/Asynchronous-Programming-NET-Richard-Blewett/dp/1430259205

class Program
{        
    static SemaphoreSlim s_sem = new SemaphoreSlim(90, 90);
    static List<Task> s_tasks = new List<Task>();   
    public static void Main()
    {                     
        for (int request = 1; request <= 1000; request++)
        {                             
            var task = FetchData();
            s_tasks.Add(task);                
        }
        Task.WaitAll(s_tasks.ToArray());
    }        
    private static async Task<string> FetchData()
    {
        try
        {
            s_sem.Wait();
            using (var wc = new MyCustomWebClient())
            {
                string content = await wc.DownloadStringTaskAsync(
                    new Uri("http://www.interact-sw.co.uk/oops/")).ConfigureAwait(continueOnCapturedContext: false);
                return content;
            }
        }
        finally
        {
            s_sem.Release(1);
        }
    }
    private class MyCustomWebClient : WebClient
    {
        protected override WebRequest GetWebRequest(Uri address)
        {
            var req = (HttpWebRequest)base.GetWebRequest(address);
            req.ServicePoint.ConnectionLimit = 30;
            return req;
        }
    }
}

问候。