Freeze-Thaw Bayesian Optimization

Kevin Swersky · Jasper Snoek · Ryan Prescott Adams

154 citations0/0 files storedarXiv (Cornell University)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Advanced Bandit Algorithms Research Machine Learning and Data Classification Gaussian Processes and Bayesian Inference

Abstract

In this paper we develop a dynamic form of Bayesian optimization for machine learning models with the goal of rapidly finding good hyperparameter settings. Our method uses the partial information gained during the training of a machine learning model in order to decide whether to pause training and start a new model, or resume the training of a previously-considered model. We specifically tailor our method to machine learning problems by developing a novel positive-definite covariance kernel to capture a variety of training curves. Furthermore, we develop a Gaussian process prior that scales gracefully with additional temporal observations. Finally, we provide an information-theoretic framework to automate the decision process. Experiments on several common machine learning models show that our approach is extremely effective in practice.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 154 citations exist upstream.

References

2012Practical Bayesian Optimization of Machine Learning Algorithms 2012Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting 2010A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning 2013Multi-Task Bayesian Optimization 2012Entropy search for information-efficient global optimization 2014Bayesian Optimization with Unknown Constraints 2008Practical Bayesian optimization

Cited by

2017Efficient Hyperparameter Optimization and Infinitely Many Armed Bandits 2019Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly 2021Bayesian Optimization in High-Dimensional Spaces: A Brief Survey 2017Fast Bayesian hyperparameter optimization on large datasets 2019Multi-fidelity Gaussian Process Bandit Optimisation

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.