我在寻找算法找到一个最佳设置的参数值。有问题的函数有很多局部极小的变化非常快。为了使事情更糟的是，测试组参数是很慢的 - 1分钟量级 - 我不能直接计算梯度

I'm looking for algorithms to find a "best" set of parameter values. The function in question has a lot of local minima and changes very quickly. To make matters even worse, testing a set of parameters is very slow - on the order of 1 minute - and I can't compute the gradient directly.

有什么著名的算法这种优化？

Are there any well-known algorithms for this kind of optimization?

我有中度的成功只是努力随机值。我想知道如果我能提高通过随机参数选择器必须采摘接近那些已产生不好的结果，在过去的参数的机会较低的性能。是否有一个名称，这个方法让我可以搜索特定的建议？

I've had moderate success with just trying random values. I'm wondering if I can improve the performance by making the random parameter chooser have a lower chance of picking parameters close to ones that had produced bad results in the past. Is there a name for this approach so that I can search for specific advice?

更多信息：

推荐答案

我已经试过模拟退火和粒子群算法。 （在此提醒您，我无法用梯度下降，因为坡度不能计算）。

I've tried Simulated Annealing and Particle Swarm Optimization. (As a reminder, I couldn't use gradient descent because the gradient cannot be computed).

我也尝试了一种算法，执行以下操作：

I've also tried an algorithm that does the following:

这正交方向是通过创建产生的随机正交矩阵的（适用的这code ），尺寸为必要的数量。

This "orthogonal direction" was generated by creating a random orthogonal matrix (adapted this code) with the necessary number of dimensions.

如果移动正交方向改进的结果，该算法只是继续与方向。如果没有一个方向改善的结果，跳跃距离被减半，一组新的正交方向将被尝试。最终，算法得出的结论一定是在一个局部最小，记住它，并重新启动了一大堆在一个新的随机点。

If moving in the orthogonal direction improved the result, the algorithm just continued with that direction. If none of the directions improved the result, the jump distance was halved and a new set of orthogonal directions would be attempted. Eventually the algorithm concluded it must be in a local minimum, remembered it and restarted the whole lot at a new random point.

此方法进行比模拟退火和粒子群相当好：它所需的（非常慢的）功能的更少的评估，以达到相同的质量的结果

This approach performed considerably better than Simulated Annealing and Particle Swarm: it required fewer evaluations of the (very slow) function to achieve a result of the same quality.

当然，我的SA和PSO的实现很可能是有缺陷的 - 这些都是棘手的算法有很大的空间调整参数。不过，我想我要提什么结束了工作最适合我。

Of course my implementations of S.A. and P.S.O. could well be flawed - these are tricky algorithms with a lot of room for tweaking parameters. But I just thought I'd mention what ended up working best for me.