In an article recently submitted to the ArXiV* server, researchers delved into the limitations of Expected Improvement (EI) in Bayesian optimization. They introduced LogEI, a novel family of acquisition functions aimed at addressing the numerical challenges that plagued EI and its variants. These LogEI functions offered improved numerical stability while maintaining or surpassing the optimization performance of traditional methods and contemporary state-of-the-art acquisition functions. The study highlighted the critical role of numerical optimization in enhancing Bayesian techniques, providing a promising avenue for more efficient optimization strategies.
Study: LogEI: Addressing Numerical Challenges in Bayesian Optimization. Image credit: Blackboard/Shutterstock
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.
Background
EI is widely used for efficiently optimizing costly-to-evaluate black-box functions, and its variants are fundamental acquisition functions in this domain. Despite their prevalence, numerical stability issues impact their performance, often resulting in suboptimal outcomes. Previous research has concentrated on optimizing algorithms and initialization strategies, neglecting the core challenge of effectively computing EI.
Enhancing Bayesian Optimization: Numerical Insights
In the theoretical analysis, the research sheds light on the conditions impacting the numerical vanishing of EI gradients. It explores the interplay between the objective function and the surrogate model. It reveals how EI exhibits vanishing gradients as Bayesian optimization (BO) algorithms narrow the gap between the global maximum, f*, of the proper function ftrue, and y*, the incumbent value.
The uncertainty reduction in the associated Gaussian process surrogate correlates with EI's gradients. The discussion focuses on the dependency of encountering numerically vanishing values and gradients in EI on the distribution of objective values f. This insight highlights how inputs yielding highly accurate values result in a slow probability decay, influencing the behavior of improvement-based acquisition functions.
The discussion extends to Monte Carlo formulations, specifically targeting challenges encountered by qEI and Quantitative Expected Hypervolume Improvement (qEHVI), where discrete maxima operations frequently result in mathematically and numerically zero gradients. Smoothing approximations and leveraging stable numerical transformations, the research introduces LogEI variants and LogEHVI, delivering numerically stable implementations that prove highly effective for optimization tasks.
These formulations significantly improve the practical implications of Bayesian optimization using EI, demonstrated by empirical results. Furthermore, the research extends LogEI formulations to handle optimization problems with black-box constraints. The research proposes analytic and Monte Carlo variants of Log constrained EI (LogCEI), showcasing how they deliver stable and accurate implementations for black-box constraint problems.
Finally, the discussion delves into the complexities of qEHVI and presents a differentiated approach to handling its numerical challenges. The implementation, leveraging differentiable inclusion-exclusion formulations, facilitates the computation of expected log hypervolume in a differentiable manner, marking a significant advancement applicable in specific multi-objective optimization scenarios. These reformulations and extensions to LogEI across various acquisition functions stand as a pivotal contribution, offering numerically stable alternatives that significantly ease optimization challenges encountered in conventional implementations, potentially advancing the efficacy of Bayesian optimization strategies.
LogEI's Superiority in Bayesian Optimization
The empirical investigation involves comparing BO techniques, including standard versions like Analytic EI and CEI, and Monte Carlo-based approaches such as qEI and qEHVI. Researchers evaluate these against state-of-the-art baselines like GIBBON and Joint Entropy Search (JES) across different test problems.
The experiments utilize multiple restarts and report mean values with standard errors for robustness. For single-objective sequential BO, the comparison focuses on a 10-dimensional convex Sum-of-Squares (SoS) function, where EI struggles due to vanishing gradients, impeding its progress. LogEI, however, demonstrates steady improvement despite this issue, showcasing its resilience in such scenarios. The study delves into performance comparisons on challenging test functions like Ackley and Michalewicz. LogEI notably outperforms EI on Ackley, particularly as the dimensionality increases. Although less pronounced, this advantage is also visible in Michalewicz, illustrating LogEI's capacity to handle functions with numerically vanishing gradients better than traditional EI.
In constrained optimization, researchers propose LogCEI for problems featuring black-box constraints. The research highlights LogCEI's superior performance over the naive CEI implementation. It even outpaces the trust-region-based SCBO method, showcasing faster convergence and improved results with increasing problem complexity and constraints. The empirical findings also discuss the advantages of qLogEI in parallel BO scenarios, especially the effectiveness of jointly optimizing batch acquisition functions compared to sequential greedy optimization, enhancing the overall optimization performance.
Extending into high-dimensional problems, LogEI demonstrates its effectiveness across various challenges, such as high-dimensional embedded functions, trajectory planning, and SVM hyperparameter tuning. Although the performance gains vary across problems, LogEI consistently presents competitive improvements. Furthermore, qLogEHVI consistently outperforms qEHVI across different batch sizes in multi-objective optimization, showcasing its superior performance in optimizing multi-objective test problems. The discussion emphasizes LogEI's advantages in handling vanishing gradients and complex optimization landscapes. It proposes its replacement or incorporation as a superior alternative to traditional EI-based approaches in Bayesian optimization scenarios.
Conclusion
To sum up, the research underscores the significant impact of vanishing gradients on improvement-based acquisition functions and demonstrates effective strategies to address this issue through careful reformulations and implementations. Modified variants of EI substantially enhance optimization performance across diverse problems and offer robustness against heuristic-dependent initialization strategies without imposing additional computational complexity.
While contributions might not be directly translatable to all acquisition function types, the insights provided could aid in improving various other acquisition functions facing similar numerical challenges. Furthermore, integrating these methods with gradient-aware first-order BO approaches could lead to highly effective applications, especially in high-dimensional search spaces. Overall, the findings emphasize the critical role of optimizing acquisition functions and the need for meticulous consideration of numerical aspects in these processes.
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.