Schedule

Subscribe to our calendar for the most recent schedule.

April 24th 1:00pm CET - Leonard Papenmeier

Title: Understanding High-Dimensional Bayesian Optimization

Abstract:

Recent work reported that simple Bayesian optimization methods perform well for high-dimensional real-world tasks, seemingly contradicting prior work and tribal knowledge. In this talk, we identify fundamental challenges in high-dimensional Bayesian optimization (HDBO) and explain why recent methods succeed. Our analysis shows that two types of vanishing gradients caused by Gaussian process (GP) initialization schemes play a major role in the failures of high-dimensional Bayesian optimization and that methods that promote local search behaviors are better suited for the task. We discuss how a simple variant of maximum likelihood estimation of GP length scales achieves state-of-the-art performance on a comprehensive set of real-world applications by leveraging these insights and discuss whether HDBO can be considered solved.