Academic Journal
One-Shot Averaging for Distributed TD(λ) Under Markov Sampling.
| Τίτλος: | One-Shot Averaging for Distributed TD(λ) Under Markov Sampling. |
|---|---|
| Συγγραφείς: | Tian H; Department of Electrical Engineering, Boston University, Boston, MA, USA., Paschalidis IC; Department of Electrical Engineering and Division of System Engineering, Boston University, Boston, MA, USA., Olshevsky A; Department of Electrical Engineering and Division of System Engineering, Boston University, Boston, MA, USA. |
| Πηγή: | IEEE control systems letters [IEEE Control Syst Lett] 2024; Vol. 8, pp. 1541-1546. Date of Electronic Publication: 2024 Jun 11. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: Institute of Electrical Engineers, Inc Country of Publication: United States NLM ID: 101708149 Publication Model: Print-Electronic Cited Medium: Print ISSN: 2475-1456 (Print) NLM ISO Abbreviation: IEEE Control Syst Lett Subsets: PubMed not MEDLINE |
| Imprint Name(s): | Original Publication: Piscataway, NJ : Institute of Electrical Engineers, Inc. |
| Περίληψη: | We consider a distributed setup for reinforcement learning, where each agent has a copy of the same Markov Decision Process but transitions are sampled from the corresponding Markov chain independently by each agent. We show that in this setting, we can achieve a linear speedup for TD(λ), a family of popular methods for policy evaluation, in the sense that N agents can evaluate a policy N times faster provided the target accuracy is small enough. Notably, this speedup is achieved by "one shot averaging," a procedure where the agents run TD(λ) with Markov sampling independently and only average their results after the final step. This significantly reduces the amount of communication required to achieve a linear speedup relative to previous work. |
| References: | Nature. 2015 May 28;521(7553):436-44. (PMID: 26017442) Nature. 2017 Oct 18;550(7676):354-359. (PMID: 29052630) IEEE Control Syst Lett. 2024;8:1541-1546. (PMID: 39950119) |
| Grant Information: | UL1 TR001430 United States TR NCATS NIH HHS |
| Contributed Indexing: | Keywords: Multi-agent system; Reinforcement Learning; Temporal Difference Learning |
| Entry Date(s): | Date Created: 20250214 Latest Revision: 20260924 |
| Update Code: | 20260924 |
| PubMed Central ID: | PMC11824493 |
| DOI: | 10.1109/LCSYS.2024.3412648 |
| PMID: | 39950119 |
| Βάση Δεδομένων: | MEDLINE |
καταχωρήστε σχόλιο πρώτοι!