Λεπτομέρειες βιβλιογραφικής εγγραφής
| Τίτλος: |
Stochastic Gradient Descent with Adaptive Data. |
| Συγγραφείς: |
Che, Ethan1 (AUTHOR) eche25@gsb.columbia.edu, Dong, Jing1 (AUTHOR) jing.dong@gsb.columbia.edu, Tong, Xin T.2 (AUTHOR) xin.t.tong@nus.edu.sg |
| Πηγή: |
Operations Research. Jul/Aug2026, Vol. 74 Issue 4, p2113-2145. 33p. |
| Θεματικοί όροι: |
*Markov processes, *Operations research, Stochastic approximation, Stability theory, Machine learning, Reinforcement learning |
| Περίληψη: |
Stochastic gradient descent (SGD) is a central tool in modern optimization, but its classical theory relies on the assumption that data are independent of the decisions being optimized. In many operations research settings, this assumption fails: policies influence system dynamics, and the resulting data feed back into subsequent updates. In "Stochastic Gradient Descent with Adaptive Data," Che, Dong, and Tong address this challenge by developing a general framework for analyzing SGD when data are generated adaptively by policy-dependent Markov processes. Their analysis shows that fully adaptive SGD can still attain convergence rates comparable to the classical i.i.d. setting, provided the underlying system satisfies mild ergodicity and continuity conditions. The theory is illustrated through canonical applications in operations research and reinforcement learning. Overall, the paper provides rigorous and reassuring theoretical foundations for deploying learning algorithms in dynamic environments where decisions and data are fundamentally intertwined. Stochastic gradient descent (SGD) is a powerful optimization technique that is particularly useful in online learning scenarios. Its convergence analysis is relatively well understood under the assumption that the data samples are independent and identically distributed (iid). However, applying SGD to policy optimization problems in operations research involves a distinct challenge: the policy changes the environment and thereby affects the data used to update the policy. The adaptively generated data stream involves samples that are nonstationary, no longer independent from each other, and affected by previous decisions. The influence of previous decisions on the data generated introduces bias in the gradient estimate, which presents a potential source of instability for online learning not present in the iid case. In this paper, we introduce simple criteria for the adaptively generated data stream to guarantee the convergence of SGD. We show that the convergence speed of SGD with adaptive data are largely similar to the classical iid setting, as long as the mixing time of the policy-induced dynamics is factored in. Our Lyapunov-function analysis allows one to translate existing stability analysis of stochastic systems studied in operations research into convergence rates for SGD, and we demonstrate this for queueing and inventory management problems. We also showcase how our result can be applied to study the sample complexity of an actor-critic policy gradient algorithm. Funding: J. Dong and E. Che acknowledge grant support from the National Science Foundation [NSF CMMI-1944209]. X. T. Tong acknowledges grant support from the Singapore Ministry of Education [MOE Grant A-8002956-00-00]. Supplemental Material: All supplemental materials, including the code, data, and files required to reproduce the results, are available at https://doi.org/10.1287/opre.2024.1014. [ABSTRACT FROM AUTHOR] |
|
Copyright of Operations Research is the property of INFORMS: Institute for Operations Research & the Management Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| Βάση Δεδομένων: |
Business Source Index |
