Regret Bounds for the Adaptive Control of Linear Quadratic Systems

Yasin Abbasi-Yadkori · Csaba Szepesvári

213 citations0/0 files stored

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Advanced Bandit Algorithms Research Advanced Control Systems Optimization Adaptive Dynamic Programming Control

Abstract

We study the average cost Linear Quadratic (LQ) control problem with unknown model parameters, also known as the adaptive control problem in the control community. We design an algorithm and prove that apart from logarithmic factors its regret up to time T is O( T). Unlike previous approaches that use a forced-exploration scheme, we construct a high-probability confidence set around the model parameters and design an algorithm that plays optimistically with respect to this confidence set. The construction of the confidence set is based on the recent results from online least-squares estimation and leads to improved worst-case regret bound for the proposed algorithm. To the best of our knowledge this is the the first time that a regret bound is derived for the LQ control problem. 1.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 213 citations exist upstream.

References

2002Finite-time Analysis of the Multiarmed Bandit Problem 2003Using confidence bounds for exploitation-exploration trade-offs 2010Near-optimal Regret Bounds for Reinforcement Learning 2008Stochastic Linear Optimization Under Bandit Feedback 2010Linearly Parameterized Bandits 2008Multi-armed bandits in metric spaces 2004Nearly Tight Bounds for the Continuum-Armed Bandit Problem

Cited by

2020Improper Learning for Non-Stochastic Control 2020Naive Exploration is Optimal for Online LQR 2012Partial Monitoring with Side Information 2018Improved Regret Bounds for Thompson Sampling in Linear Quadratic Control Problems 2014Dynamic Pricing with Multiple Products and Partially Specified Demand Distribution 2012The adversarial stochastic shortest path problem with unknown transition probabilities 2019Learning Linear-Quadratic Regulators Efficiently with only $\sqrt{T}$ Regret

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.