Closed-loop control with delayed information

Abstract

The theory of Markov Control Model with Perfect State Information (MCM-PSI) requires that the current state of the system is known to the decision maker at decision instants. Otherwise, one speaks of Markov Control Model with Imperfect State Information (MCM-ISI). In this article, we introduce a new class of MCM-ISI, where the information on the state of the system is delayed. Such an information structure is encountered, for instance, in high-speed data networks. In the first part of this article, we show that by enlarging the state space so as to include the last known state as well as all the decisions made during the travel time of the information, we may reduce a MCM-ISI to a MCM-PSI. In the second part of this paper, this result is applied to a flow control problem. Considered is a discrete time queueing model with Bernoulli arrivals and geometric services, where the intensity of the arrival stream is controlled. At the beginning of slot t +1, t =0,1,2,…, the decision maker has to select the probability of having one arrival in the current time slot from the set { p 1 , p 2 }, 0 ≤ p 2 < p 1 ≤ 1, only on the basis of the queue-length and action histories in [ 0, t ]. The aim is to optimize a discounted throughput/delay criterion. We show that there exists an optimal policy of a threshold type, where the threshold is seen to depend on the last action.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 40 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.