Design and analysis of simulations: practical statistical techniques

Abstract

Using elementary statistical theory, we discuss sta tistical techniques that can be used to initialize a simulation run and to determine its length, distin guishing between terminating and nonterminating sys tems and between stationary and nonstationary time series. Confidence intervals and hypothesis tests are included (see Section 2). In the case of k sys tem variants (at least 2), multiple comparison proce dures are presented which can be used to obtain simultaneously valid confidence intervals and to select a subset containing the best population, assuming a fixed number of simulation runs. Other wise ranking procedures can be used to determine the number of runs required to select the best system (Section 3). If many parameters and variables exist, selecting a limited number of combinations requires an experimental design, which must be analyzed with a regression metamodel (Section 4). The metamodel of main effects and interactions applies to both quantitative and qualitative factors; its adequacy can be tested (Subsection 4.1). Experimental design is discussed in three steps: (1) screening designs for finding the important factors, namely, 2 k-P designs, random, supersaturated, and group-screening designs (Subsection 4.2); (2) augmented 2 K-P designs for further exploration (Subsection 4.3); (3) response surface methodology for optimization (Subsection 4.4). Three practical variance reduction techniques are summarized, namely, control variates, antithetics, and common random numbers (Section 5). Some of the statistical methods discussed are also applicable to deterministic models of continuous systems. This paper was written for readers with only an elementary background in statistics.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 28 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.