如何限制Parallel.ForEach?Parallel、ForEach

2023-09-02 01:29:18 作者:敌敌畏买一送一

我有一个Parallel.ForEach()异步环路与我下载了一些网页。我的带宽是有限的,所以我能下载只有X页每一次,但Parallel.ForEach执行所需的网页的完整列表。

I have a Parallel.ForEach() async loop with which I download some webpages. My bandwidth is limited so I can download only x pages per time but Parallel.ForEach executes whole list of desired webpages.

有没有办法来限制线程号或其他任何限制,同时运行Parallel.ForEach?

Is there a way to limit thread number or any other limiter while running Parallel.ForEach?

演示code:

Parallel.ForEach(listOfWebpages, webpage => {
  Download(webpage);
});

真正的任务有无关的网页,太有创意了网页抓取的解决方案也无济于事。

The real task has nothing to do with webpages, so creative web crawling solutions won't help.

推荐答案

您可以指定一个 MaxDegreeOfParallelism ParallelOptions 参数:

You can specify a MaxDegreeOfParallelism in a ParallelOptions parameter:

Parallel.ForEach(
    listOfWebpages,
    new ParallelOptions { MaxDegreeOfParallelism = 4 },
    webpage => { Download(webpage); }
);

MSDN: Parallel.ForEach

MSDN:ParallelOptions.MaxDegreeOfParallelism