A Restless Bandit Formulation of Opportunistic Access: Indexablity and Index Policy

Abstract

We focus on an opportunistic communication system consisting of multiple independent channels with time-varying states. With limited sensing, a user can only sense and access a subset of channels and accrue rewards determined by the state of the sensed channels. We formulate the problem of optimal sequential channel probing as a restless multi-armed bandit process, for which a powerful index policy-Whittle's index policy-can be implemented based on the indexability of the system. Exploiting the underlying structure of the multi-channel opportunistic access problem, we establish the indexability and obtain the Whittle's index in closed-form, which leads to a direct implementation of Whittle's index policy with little complexity. Furthermore, we show that Whittle's index policy is equivalent to the myopic policy when channels are statistically identical.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 51 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.