Do you have a noisy black-box with a number of parameters that need to be tuned? Then you might just be in luck! The newest paper of the well-known researcher, Rémi Coulom, might be just what you are looking for. In “CLOP: Confident Local Optimization for Noisy Black-Box Parameter Tuning” he describes a new method for optimizing a number of black-box parameters given a noisy output.
To be honest, I have only glanced over the theory behind it, but I have already witnessed some of its power. The reason that is possible is that Rémi has also released software to use. His software is open source and runs on Linux, Mac and Windows. After downloading and building the software, you can run the GUI and be greeted with a simple and intuitive interface. I have included a few screenshots, so you can see what the provided example looks like.
If you think that this is relevant to your work, why not give it a chance? I know I will be making use of it to tune my Computer Go program, Oakfoam. My only complaint is that, in my opinion, the command line interface is slightly lacking; then again, the code is open source, so there is nothing stopping me from improving it…
For more info: http://remi.coulom.free.fr/CLOP/