Theadaptivecriticheuristichasbeenapopularalgorithminreinforcementlearning(RL)andapproximatedynamicprogramming(ADP)alike.ItisoneofthefirstRLandADPalgorithms.RLandADPalgorithmsareparticularlyusefulforsolvingMarkovdecisionprocesses(MDPs)thatsufferfromthecursesofdimensionalityandmodeling.Manyreal-worldproblems,however,tendtobesemi-Markovdecisionprocesses(SMDPs)inwhichthetimespentineachtransitionoftheunderlyingMarkovchainsisitselfarandomvariab...