A Stochastic Approximation Frame Algorithm with Adaptive Directions

(整期优先)网络出版时间:2008-04-14
/ 1
Stochasticapproximationproblemistofindsomerootorextremumofanon-linearfunctionforwhichonlynoisymeasurementsofthefunctionareavailable.TheclassicalalgorithmforstochasticapproximationproblemistheRobbins-Monro(RM)algorithm,whichusesthenoisyevaluationofthenegativegradientdirectionastheiterativedirection.InordertoacceleratetheRMalgorithm,thispapergivesaflamealgorithmusingadaptiveiterativedirections.Ateachiteration,thenewalgorithmgoestowardseitherthenoisyevaluationofthenegativegradientdirectionorsomeotherdirectionsundersomeswitchcriterions.Twofeasiblechoicesofthecriterionsarepro-posedandtwocorrespondingflamealgorithmsareformed.Differentchoicesofthedirectionsunderthesamegivenswitchcriterionintheflamecanalsoformdifferentalgorithms.Wealsoproposedthesimultanousperturbationdifferenceformsforthetwoflamealgorithms.Thealmostsurelyconvergenceofthenewalgorithmsareallestablished.Thenumericalexperimentsshowthatthenewalgorithmsarepromising.