Weighted Means in Stochastic Approximation of Minima

Julia Dippon · Jessica Renz

91 citations0/0 files storedSIAM Journal on Control and Optimization

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Simulation Techniques and Applications Stochastic Gradient Optimization Techniques Extremum Seeking Control Systems

Abstract

Weighted averages of Kiefer--Wolfowitz-type procedures, which are driven by larger step lengths than usual, can achieve the optimal rate of convergence. A priori knowledge of a lower bound on the smallest eigenvalue of the Hessian matrix is avoided. The asymptotic mean squared error of the weighted averaging algorithm is the same as would emerge using a Newton-type adaptive algorithm. Several different gradient estimates are considered; one of them leads to a vanishing asymptotic bias. This gradient estimate applied with the weighted averaging algorithm usually yields a better asymptotic mean squared error than applied with the standard algorithm.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 91 citations exist upstream.

References

1992Multivariate stochastic approximation using a simultaneous perturbation gradient approximation 1952Stochastic Estimation of the Maximum of a Regression Function 1992Acceleration of Stochastic Approximation by Averaging 1978Stochastic Approximation Methods for Constrained and Unconstrained Systems 2005STOCHASTIC APPROXIMATION 1954Multidimensional Stochastic Approximation Methods 1968On Asymptotic Normality in Stochastic Approximation 1967Stochastic Approximation of Minima with Improved Asymptotic Speed

Cited by

2008Global Random Optimization by Simultaneous Perturbation Stochastic Approximation 2005Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization 2007A companion for the Kiefer–Wolfowitz–Blum stochastic approximation algorithm 2003Convergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization 2013Simulation Optimization: A Concise Overview and Implementation Guide

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.