Surveillance in an abruptly changing world via multiarmed bandits

Vaibhav Srivastava · Paul B. Reverdy · Naomi Ehrich Leonard

34 citations0/0 files storedIEEE Conference on Decision and Control (CDC)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Advanced Bandit Algorithms Research Reinforcement Learning in Robotics Optimization and Search Problems

Abstract

We study a path planning problem in an environment that is abruptly changing due to the arrival of unknown spatial events. The objective of the path planning problem is to collect the data that is most evidential about the events. We formulate this problem as a multiarmed bandit (MAB) problem with Gaussian rewards and change points, and address the fundamental tradeoff between learning the true event (exploration), and collecting the data that is most evidential about the true event (exploitation). We extend the switching-window UCB algorithm for MAB problems with bounded rewards and change points to the context of correlated Gaussian rewards and develop the switching-window UCL (SW-UCL) algorithm. We extend the SW-UCL algorithm to an adaptive SW-UCL algorithm that utilizes statistical change detection to adapt the SW-UCL algorithm. We also develop a block SW-UCL algorithm that reduces the number of transitions among arms in the SW-UCL algorithm, and is more amenable to robotic applications.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 34 citations exist upstream.

References

2002Finite-time Analysis of the Multiarmed Bandit Problem 2012Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems 2012Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems 1970Inference About the Change-Point in a Sequence of Random Variables 2012On Bayesian Upper Confidence Bounds for Bandit Problems 2008On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems 1988Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching cost 2014Modeling Human Decision Making in Generalized Gaussian Multiarmed Bandits

Cited by

2018On Abruptly-Changing and Slowly-Varying Multiarmed Bandit Problems 2018Social Imitation in Cooperative Multiarmed Bandits: Partition-Based Algorithms with Strictly Local Information 2016Satisficing in Multi-Armed Bandit Problems

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.