Academic Journal
Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for safe reinforcement learning in optimal control.
| Title: | Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for safe reinforcement learning in optimal control. |
|---|---|
| Authors: | Khalili-Amirabadi R; Department of Applied Mathematics, Ferdowsi University of Mashhad, Mashhad, Iran. roya.khalili.a@gmail.com., Jalaeian-Farimani M; Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Milan, Italy., Solaymani-Fard O; Department of Applied Mathematics, Ferdowsi University of Mashhad, Mashhad, Iran. |
| Source: | Scientific reports [Sci Rep] 2026 Mar 25; Vol. 16 (1). Date of Electronic Publication: 2026 Mar 25. |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Nature Publishing Group Country of Publication: England NLM ID: 101563288 Publication Model: Electronic Cited Medium: Internet ISSN: 2045-2322 (Electronic) Linking ISSN: 20452322 NLM ISO Abbreviation: Sci Rep Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: London : Nature Publishing Group, copyright 2011- |
| MeSH Terms: | Clustering Algorithms* , Reinforcement Machine Learning*, Humans ; Adaptive Algorithms ; Cluster Analysis ; Nonlinear Dynamics ; Soft Computing |
| Abstract: | This paper proposes a novel reinforcement learning framework, named Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER), designed to achieve safe and scalable optimal control of nonlinear systems. The proposed SODACER mechanism consists of a Fast-Buffer for rapid adaptation to recent experiences and a Slow-Buffer equipped with a self-organizing adaptive clustering mechanism to maintain diverse and non-redundant historical experiences. The adaptive clustering mechanism dynamically prunes redundant samples, optimizing memory efficiency while retaining critical environmental patterns. The approach integrates SODACER with Control Barrier Functions (CBFs) to guarantee safety by enforcing state and input constraints throughout the learning process. To enhance convergence and stability, the framework is combined with the Sophia optimizer, enabling adaptive second-order gradient updates. The proposed SODACER-Sophia's architecture ensures reliable, effective, and robust learning in dynamic, safety-critical environments, offering a generalizable solution for applications in robotics, healthcare, and large-scale system optimization. The proposed approach is validated on a nonlinear Human Papillomavirus (HPV) transmission model with multiple control inputs and safety constraints. Comparative evaluations against random and clustering-based experience replay methods demonstrate that SODACER achieves faster convergence, improved sample efficiency, and a superior bias-variance trade-off, while maintaining safe system trajectories, validated via the Friedman test. (© 2026. The Author(s).) |
| Competing Interests: | Declarations. Competing interests: The authors declare no competing interests. |
| References: | Math Biosci Eng. 2013 Aug;10(4):1173-205. (PMID: 23906207) IEEE Trans Neural Netw Learn Syst. 2022 Jul;33(7):2781-2790. (PMID: 33417569) ISA Trans. 2026 Jan;168:465-478. (PMID: 41309415) J Theor Biol. 2016 Mar 21;393:179-93. (PMID: 26796222) Nature. 2015 Feb 26;518(7540):529-33. (PMID: 25719670) Math Biosci. 2011 Jun;231(2):126-34. (PMID: 21377481) |
| Contributed Indexing: | Keywords: Adaptive clustering; Dual-buffer experience replay; HPV model; Nonlinear optimal control; Safe reinforcement learning |
| Entry Date(s): | Date Created: 20260326 Date Completed: 20260713 Latest Revision: 20260813 |
| Update Code: | 20260813 |
| PubMed Central ID: | PMC13168364 |
| DOI: | 10.1038/s41598-026-44517-1 |
| PMID: | 41882163 |
| Database: | MEDLINE |
Be the first to leave a comment!