我想描述我的计划的某些细节并获得最佳的多线程模型中使用，这将是最适用的反馈。我花了很多时间，现在阅读线程池，线程，生产者/消费者等，还没有得出可靠的结论。

I would like to describe some specifics of my program and get feedback on what the best multithreading model to use would be most applicable. I've spent a lot of time now reading on ThreadPool, Threads, Producer/Consumer, etc. and have yet to come to solid conclusions.

予有文件的列表（所有相同的格式），但具有不同的内容。我要对每个文件进行工作。这项工作包括读取文件，一些处理大约需要1-2分钟的直接数字运算，然后写大的输出文件的结尾。

I have a list of files (all the same format) but with different contents. I have to perform work on each file. The work consists of reading the file, some processing that takes about 1-2 minutes of straight number crunching, and then writing large output files at the end.

我想的UI界面仍然响应后，我开始了工作，对指定的文件。

I would like the UI interface to still be responsive after I initiate the work on the specified files.

几个问题：

更新：

很好的意见，到目前为止，非常有帮助。我加入了一些更细节如下要求：

Great feedback so far, very helpful. I'm adding some more details that are asked below:

输出是将多个独立的文件。一组是那么自己得到了工作项目之前读取和另一个进程处理每个工作项目输出文件完成

Output is to multiple independent files. One set of output files per "work item" that then themselves gets read and processed by another process before the "work item" is complete

工作项/线程不共享任何资源。

The work items/threads do not share any resources.

这些工作项目部分使用非托管静态库，使得使用Boost库的处理。

The work items are processed in part using a unmanaged static library that makes use of boost libraries.

推荐答案

根据意见的更新： 我不同意这一说法，一个线程池将无法处理你遇到的工作量......让我们来看看您的问题，并得到更具体： 1.你有差不多1000个文件。 2.每个文件可能需要长达2分钟的CPU密集型工作流程。 3.您想拥有并行处理来提高吞吐量。 4.你想发出信号时，每个文件是完整的，更新的用户界面。

Update based on comments: I don't agree with the statement that a ThreadPool will not be able to handle the workload you're encountering... let's look at your problem and get more specific: 1. You have almost 1000 files. 2. Each file might take up to 2 minutes of CPU-intensive work to process. 3. You want to have parallel processing to increase throughput. 4. You want to signal when each file is complete and update the UI.

实事求是你不想跑千线程，因为你用的内核数量限制你......并且由于它是CPU密集型的工作，你很可能会最大程度的发挥CPU的负载非常少的线程（在我的程序它通常是最优的每个核心有2-4个线程）。

Realistically you don't want to run 1000 threads, because you're limited by the number of cores you have... and since it's CPU intensive work you are likely to max out the CPU load with very few threads (in my programs it's usually optimal to have 2-4 threads per core).

所以，你应该不加载在线程池1000的工作项目，希望看到增加吞吐量。你必须要创造一个你总是与线程的最佳数量运行环境，这就需要一些工程。

So you shouldn't load 1000 work items in the ThreadPool and expect to see an increase of throughput. You'll have to create an environment where you're always running with an optimal number of threads and this requires some engineering.

我要反驳我原来的说法有点和实际建议生产者/消费者设计。看看这个question为上的图案的更多细节。

I'll have to contradict my original statement a little bit and actually recommend a Producer/Consumer design. Check out this question for more details on the pattern.

下面是制片人可能看起来是这样的：

Here is what the Producer might look like:

class Producer
{
    private final CountDownLatch _latch;
    private final BlockingQueue _workQueue;
    Producer( CountDownLatch latch, BlockingQueue workQueue)
    {
        _latch = latch;
        _workQueue = workQueue;
    }

    public void Run()
    {
        while(hasMoreFiles)
        {
            // load the file and enqueue it
            _workQueue.Enqueue(nextFileJob);
        }

        _latch.Signal();
    }
}

下面是你的消费者：

class Consumer
{
    private final CountDownLatch _latch;
    private final BlockingQueue _workQueue;

    Consumer(CountDownLatch latch, BlockingQueue workQueue, ReportStatusToUI reportDelegate)
    {
        _latch = latch;
        _workQueue = workQueue;
    }

    public void Run()
    {
        while(!terminationCondition)
        {
            // blocks until there is something in the queue
            WorkItem workItem = _workQueue.Dequeue();

            // Work that takes 1-2 minutes
            DoWork(workItem);

            // a delegate that is executed on the UI (use BeginInvoke on the UI)
            reportDelegate(someStatusIndicator);
        }

        _latch.Signal();
    }
}

A CountDownLatch ：

public class CountDownLatch
{
    private int m_remain;
    private EventWaitHandle m_event;

    public CountDownLatch(int count)
    {
        Reset(count);
    }

    public void Reset(int count)
    {
        if (count < 0)
            throw new ArgumentOutOfRangeException();
        m_remain = count;
        m_event = new ManualResetEvent(false);
        if (m_remain == 0)
        {
            m_event.Set();
        }
    }

    public void Signal()
    {
        // The last thread to signal also sets the event.
        if (Interlocked.Decrement(ref m_remain) == 0)
            m_event.Set();
    }

    public void Wait()
    {
        m_event.WaitOne();
    }
}

Jicksa的的BlockingQueue ：

class BlockingQueue<T> {
    private Queue<T> q = new Queue<T>();

    public void Enqueue(T element) {
        q.Enqueue(element);
        lock (q) {
            Monitor.Pulse(q);
        }
    }

    public T Dequeue() {
        lock(q) {
            while (q.Count == 0) {
                Monitor.Wait(q);
            }
            return q.Dequeue();
        }
    }
}

那么，这是否离开？现在好了，所有你所要做的就是启动所有的线程... 您可以在线程池，因为的BackgroundWorker ，或者每一个为新帖的，它并没有什么区别的

void StartThreads()
{
    CountDownLatch latch = new CountDownLatch(numConsumer+numProducer);
    BlockingQueue<T> workQueue = new BlockingQueue<T>();

    Producer producer = new Producer(latch, workQueue);
    if(youLikeThreads)
    {
        Thread p = new Thread(producer.Run);
        p.IsBackground = true;
        p.Start();
    }
    else if(youLikeThreadPools)
    {
        ThreadPool.QueueUserWorkItem(producer.Run);
    }

    for (int i; i < numConsumers; ++i)
    {
        Consumer consumer = new Consumer(latch, workQueue, theDelegate);

        if(youLikeThreads)
        {
            Thread c = new Thread(consumer.Run);

            c.IsBackground = true;

            c.Start();
        }
        else if(youLikeThreadPools)
        {
            ThreadPool.QueueUserWorkItem(consumer.Run);
        }
    }

    // wait for all the threads to signal
    latch.Wait();

    SayHelloToTheUI();
}