我已经得到了重新的尺寸图像的Web应用程序。重新大小的图像被写入磁盘，以便对它们进行缓存。什么是prevent多，从产生相同的图像的并发请求的最佳方法是什么？

I've got a web application that re-sizes images. The re-sized images are written to disk in order to cache them. What is the best way to prevent multiple, simultaneous requests from generating the same image?

有两件事情需要注意，我们有几百万的图像（以TB计）。尚未在一段时间观察缓存的图像被除去。我们有一个Web场，但每个Web服务器都有它自己的本地缓存（原件保存在另一个服务器上）。我们也将重定尺寸的图像在一二线缓存一旦被产生，以便其他Web服务器可以检查有看到，如果图像被高速缓存，如果是，则本地复制

A couple things to note, we have millions of images (measured in terabytes). Cached images that haven't been viewed in a while are removed. We have a web farm, but each web server has it's own local cache (the originals are stored on another server). We also place the re-sized images in a second-tier cache once they are generated so other web servers can check there to see if the image is cached, if it is, it is copied local.

我用锁考虑过（我贴一类使用我考虑here).但是，这显然不会与第二层高速缓存工作，我不知道，如果它是在普通的Web服务器使用的锁在一个不错的主意（虽然我不知道为什么，只是一堆模糊的提到的它是一个坏主意）。

I've considered using locks (I posted a class that I'm considering using here). But that obviously won't work with the 2nd-tier cache and I'm not sure if it is a good idea in general on a web server to use locks (though I'm not sure why, just a bunch of vague references to it being a bad idea).

我也考虑写一个临时文件，我可以检查之前，我开始创建图像，但我担心，Windows将不会清理文件正确的时间100％（锁定问题，等等）。

I've also considered writing a temp file that I could check before I start creating the image, but I'm concerned that Windows won't clean up the file properly 100% of the time (locking issues, etc).

任何想法是AP preciated。

Any ideas are appreciated.

推荐答案

您可以将操作系统互斥和原子FS重命名操作用于这一目的。这里是一个伪code：

You can combine OS mutexes and atomic FS rename operations for that purpose. Here is a pseudo code:

byte[] getResizedImage(string fileName)
{
    byte[] result;
    if(File.Exists(fileName)) return File.ReadAllBytes(fileName);
    bool created;
    using(var mutex = new Mutex(true, "Brian_ResizeCache_" + fileName, out created)) {
        if(!created) mutex.WaitOne();
        // re-check: file could have been written just when 
        // we were waiting for the mutex
        if(File.Exists(fileName)) { 
            mutex.ReleaseMutex(); // release early 
            return File.ReadAllBytes(fileName);
        }
        result = retrieveFromSecondTierCache(fileName);
        if(result == null) { 
            // try central server and retrieve
            result = retrieveFromCentralServer(fileName);
            result = resizeImage(result);
            saveToSecondTierCache(fileName, result);
        }
        string tempFileName = generateLocalTempFile(result);
        File.RenameFile(tempFileName, fileName);
        mutex.ReleaseMutex();
        return result;
    }
}

注：

在我没有测试过这种方法，但它应该没问题。对于已经创建的缓存的内容，你甚至不需要处理锁。 在这两个二线接入和调整工作被认为是生成本地缓存过程的一部分。 如果一个线程开始产生本地缓存文件，要求相同的其他线程将等待该线程完成，它们将消耗没有CPU，直到生成完成。重要的是你知道CPU是因为这种类型的锁等待也可以填写服务器积压容易。的瓶颈 这将在多个流程/应用程序池的工作，由于命名互斥的本质。 ReleaseMutex（）可能没有必要，我希望使用块将摆脱它。 在每个文件会占用图像生成过程中的OS互斥表中的单个项目。具有最高的并行性，但使用最多的资源。如果原来是太多了，你可以用每个桶互斥量，而不是像 fileName.GetHash code（）％numBuckets一个单一的文件等 这也可以转换为使用FS锁定，而不是互斥。这将是慢，但能更好地扩展，这取决于每秒等即时生成操作。 I haven't tested this approach but it should be ok. For already created cache content you don't even need to deal with locks. Both 2nd tier access and resizing work is considered part of "generating local cache" process. If one thread starts generating local cache file, other threads requesting the same will be waiting for this thread to finish, they will consume no CPU until the generation is complete. It's important that you know the CPU is the bottleneck since this type of "lock-waiting" can also fill the server backlog easily. This will work across multiple processes/app pools due to nature of named mutexes. ReleaseMutex() may not be necessary, I'd hope the using block would get rid of it. Each file will occupy a single entry in the OS mutex table during image generation. That has the highest parallelism but uses most resources. If that turns out to be too many, you can sacrifice granularity by using "bucket" mutexes instead of each for a single file like fileName.GetHashCode() % numBuckets etc. This could also be converted to use FS locking instead of mutexes. It would be slower but could scale better, depending on immediate generation operations per second etc.

总之，这种优化的成功主要取决于运行参数别人提及。同时请求，缓存命中率，存储时间花在生成本地缓存的随时间的数花在仔细调整等基准，这仅仅是一个模板。

To summarize, the success of such optimizations mostly depend on runtime parameters as others mentioned. Number of simultaneous requests, cache hit ratio, storage, time spent on generation of local cache vs time spent on resizing etc. Benchmark carefully, this is just a template.