On the public citation graph

The actor-critic algorithm as multi-time-scale stochastic approximation

https://doi.org/10.1007/bf02745577
0 signed · 49 inferred · view as graph

Inferred edges come from two machine sources: sweeps of open repository metadata (mostly SSRN and Zenodo), and a bulk OpenAlex citation snapshot. Every row below is labelled with the one it came from. Neither is a complete citation index — absence here is not evidence of absence.

Is this your work? Sign in with ORCID and put your name on the claims that are yours.
Claim with ORCID
Reference health. See whether every reference in this work resolves to a known work and none has been retracted — and get a dated CiteStamped badge you can embed.
See reference health

What this work says

Outgoing claims: the papers this work cites, builds on, or argues with.

inferredcites →https://doi.org/10.1002/9780470316887openalex-snapshot
Markov Decision Processes
inferredcites →https://doi.org/10.1007/978-0-387-35434-7_14openalex-snapshot
Discrete Parameter Martingales
inferredcites →https://doi.org/10.1007/978-1-4684-9352-8openalex-snapshot
Stochastic Approximation Methods for Constrained and Unconstrained Systems
inferredcites →https://doi.org/10.1007/978-3-642-75894-2openalex-snapshot
Adaptive Algorithms and Stochastic Approximations
inferredcites →https://doi.org/10.1007/bf00114724openalex-snapshot
Feature-based methods for large scale dynamic programming
inferredcites →https://doi.org/10.1007/bf02743935openalex-snapshot
A tutorial survey of reinforcement learning
inferredcites →https://doi.org/10.1016/0004-3702(94)00011-oopenalex-snapshot
Learning to act using real-time dynamic programming
inferredcites →https://doi.org/10.1016/0010-4485(76)90048-8openalex-snapshot
IEEE trans. syst. man cybern
inferredcites →https://doi.org/10.1016/0022-247x(85)90317-8openalex-snapshot
Generalized polynomial approximations in Markovian decision processes
inferredcites →https://doi.org/10.1016/0024-3795(69)90028-7openalex-snapshot
Chaotic relaxation
inferredcites →https://doi.org/10.1016/0893-6080(89)90018-xopenalex-snapshot
Convergent activation dynamics in continuous time networks
inferredcites →https://doi.org/10.1016/s0167-6911(97)90015-3openalex-snapshot
Stochastic approximation with two time scales
inferredcites →https://doi.org/10.1017/s0021900200030825openalex-snapshot
Learning algorithms for Markov decision processes
inferredcites →https://doi.org/10.1023/a:1022676722315openalex-snapshot
Technical Note: Q-Learning
inferredcites →https://doi.org/10.1023/a:1022689125041openalex-snapshot
Asynchronous Stochastic Approximation and Q-Learning
inferredcites →https://doi.org/10.1057/jors.1990.169openalex-snapshot
An Introduction to Queueing Networks
inferredcites →https://doi.org/10.1080/17442508708833435openalex-snapshot
Estimation and control in discounted stochastic dynamic programming
inferredcites →https://doi.org/10.1109/3468.618258openalex-snapshot
A reinforcement learning neural network for adaptive control of Markov chains
inferredcites →https://doi.org/10.1109/tsmc.1983.6313077openalex-snapshot
Neuronlike adaptive elements that can solve difficult learning control problems
inferredcites →https://doi.org/10.1112/jlms/s1-39.1.382aopenalex-snapshot
THEORY AND APPLICATION OF LIAPUNOV'S DIRECT METHOD
inferredcites →https://doi.org/10.1214/aop/1176990853openalex-snapshot
Nonconvergence to Unstable Points in Urn Models and Stochastic Approximations
inferredcites →https://doi.org/10.17863/cam.132164openalex-snapshot
Learning from Delayed Rewards
inferredcites →https://doi.org/10.2307/2344365openalex-snapshot
Discrete-Parameter Martingales.

What points at this work

Incoming claims: where this work appears in other papers’ reference lists and arguments.

inferred← citeshttps://doi.org/10.1002/9781118029176.biblioopenalex-snapshot
Bibliography
inferred← citeshttps://doi.org/10.1002/9781119815068.ch17openalex-snapshot
Forward ADP II : Policy Optimization
inferred← citeshttps://doi.org/10.1007/s10626-006-0003-yopenalex-snapshot
Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes
inferred← citeshttps://doi.org/10.1007/s11227-022-04643-9openalex-snapshot
Q-learning-based algorithms for dynamic transmission control in IoT equipment
inferred← citeshttps://doi.org/10.1016/j.automatica.2025.112395openalex-snapshot
An approximate policy iteration viewpoint of actor–critic algorithms
inferred← citeshttps://doi.org/10.1016/j.eswa.2026.133705openalex-snapshot
DQNEGHF: Integrating deep Q-networks with evolutionary game theory for dynamic beam-hopping in spectrum-sharing satellite networks
inferred← citeshttps://doi.org/10.1016/s0005-1098(02)00231-5openalex-snapshot
Self-learning control of finite Markov chains
inferred← citeshttps://doi.org/10.1017/9781009028943.022openalex-snapshot
Reinforcement Learning for Mean Field Games, with Applications to Economics
inferred← citeshttps://doi.org/10.1109/access.2021.3103767openalex-snapshot
Multi-Time Scale Smoothed Functional With Nesterov’s Acceleration
inferred← citeshttps://doi.org/10.1109/cdc56724.2024.10886557openalex-snapshot
A System-theoretic Note on Model-Based Actor-Critic
inferred← citeshttps://doi.org/10.1109/ciss53076.2022.9751163openalex-snapshot
Policy Gradient for Ratio Optimization: A Case Study
inferred← citeshttps://doi.org/10.1109/jsac.2010.100403openalex-snapshot
A systematic framework for dynamically optimizing multi-user wireless video transmission
inferred← citeshttps://doi.org/10.1109/lcsys.2022.3172242openalex-snapshot
Finite-Sample Analysis of Off-Policy Natural Actor–Critic With Linear Function Approximation
inferred← citeshttps://doi.org/10.1109/tac.2020.3029315openalex-snapshot
Policy Evaluation in Continuous MDPs With Efficient Kernelized Gradient Temporal Difference
inferred← citeshttps://doi.org/10.1109/tsp.2023.3268475openalex-snapshot
Towards Understanding Asynchronous Advantage Actor-Critic: Convergence and Linear Speedup
inferred← citeshttps://doi.org/10.1137/s036301299731669xopenalex-snapshot
Actor-Critic--Type Learning Algorithms for Markov Decision Processes
inferred← citeshttps://doi.org/10.1214/25-sts999openalex-snapshot
Stochastic Approximation and Reinforcement Learning: The Interface and a Little Beyond
inferred← citeshttps://doi.org/10.26686/wgtn.17138678openalex-snapshot
Policy Direct Search for Effective Reinforcement Learning
inferred← citeshttps://doi.org/10.26686/wgtn.17138678.v1openalex-snapshot
Policy Direct Search for Effective Reinforcement Learning
inferred← citeshttps://doi.org/10.4208/jml.230919openalex-snapshot
Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces
inferred← citeshttps://doi.org/10.48550/arxiv.1902.00183openalex-snapshot
Learning Action Representations for Reinforcement Learning
inferred← citeshttps://doi.org/10.48550/arxiv.1907.06246openalex-snapshot
On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost
inferred← citeshttps://doi.org/10.48550/arxiv.2005.01350openalex-snapshot
A Finite Time Analysis of Two Time-Scale Actor Critic Methods
inferred← citeshttps://doi.org/10.48550/arxiv.2008.00483openalex-snapshot
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy
inferred← citeshttps://doi.org/10.48550/arxiv.2103.04021openalex-snapshot
Asymptotic Theory for IV-Based Reinforcement Learning with Potential Endogeneity
inferred← citeshttps://doi.org/10.48550/arxiv.2108.08655openalex-snapshot
Global Convergence of the ODE Limit for Online Actor-Critic Algorithms in Reinforcement Learning
Editorial review for independent work. ICSAC — the Institute for Complexity Science and Advanced Computing — runs a structured editorial panel for researchers publishing outside journals, and publishes on CiteStamp rails. Submission is free and goes through ORCID: icsacinstitute.org/submit.