Academic Journal
Προβολή σε EDS

Square-Root Regret Bounds for Continuous-Time Episodic Markov Decision Processes.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Square-Root Regret Bounds for Continuous-Time Episodic Markov Decision Processes.
Συγγραφείς: Gao, Xuefeng1 (AUTHOR) xfgao@se.cuhk.edu.hk, Zhou, Xunyu2 (AUTHOR) xz2574@columbia.edu
Πηγή: Mathematics of Operations Research (INFORMS). Feb2026, Vol. 51 Issue 1, p333-357. 25p.
Θεματικοί όροι: *Markov processes, *Dynamic programming, *Simulation methods & models, Reinforcement learning, Confidence intervals
Περίληψη: We study reinforcement learning for continuous-time Markov decision processes (MDPs) in the finite-horizon episodic setting. In contrast to discrete-time MDPs, the intertransition times of a continuous-time MDP are exponentially distributed with rate parameters depending on the state–action pair at each transition. We present a learning algorithm based on the methods of value iteration and upper confidence bound. We derive an upper bound on the worst case expected regret for the proposed algorithm and establish a worst case lower bound with both bounds of the order of square root on the number of episodes. Finally, we conduct simulation experiments to illustrate the performance of our algorithm. Funding: X. Gao is supported by the Hong Kong Research Grant Council [Grants 14201421, 14212522, 14200123]. X. Zhou gratefully acknowledges financial support through the Nie Center for Intelligent Asset Management at Columbia. [ABSTRACT FROM AUTHOR]
Copyright of Mathematics of Operations Research (INFORMS) is the property of INFORMS: Institute for Operations Research & the Management Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Βάση Δεδομένων: Business Source Index
FullText Text:
  Availability: 0
CustomLinks:
  – Url: http://www.jstor.org/openurl?issn=0364765X&date=2026&volume=51&issue=1&spage=333
    Name: JSTOR (all content) (s7799221)
    Category: fullText
    Text:
    Icon: https://imageserver.ebscohost.com/branding/jstor/iconJSTOR.gif
    MouseOverText: Full Text from JSTOR
Header DbId: bsx
DbLabel: Business Source Index
An: 191729069
RelevancyScore: 1470
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1470.35424804688
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Square-Root Regret Bounds for Continuous-Time Episodic Markov Decision Processes.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Gao%2C+Xuefeng%22">Gao, Xuefeng</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> xfgao@se.cuhk.edu.hk</i><br /><searchLink fieldCode="AR" term="%22Zhou%2C+Xunyu%22">Zhou, Xunyu</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> xz2574@columbia.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Mathematics+of+Operations+Research+%28INFORMS%29%22">Mathematics of Operations Research (INFORMS)</searchLink>. Feb2026, Vol. 51 Issue 1, p333-357. 25p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Markov+processes%22">Markov processes</searchLink><br />*<searchLink fieldCode="DE" term="%22Dynamic+programming%22">Dynamic programming</searchLink><br />*<searchLink fieldCode="DE" term="%22Simulation+methods+%26+models%22">Simulation methods & models</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+intervals%22">Confidence intervals</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We study reinforcement learning for continuous-time Markov decision processes (MDPs) in the finite-horizon episodic setting. In contrast to discrete-time MDPs, the intertransition times of a continuous-time MDP are exponentially distributed with rate parameters depending on the state–action pair at each transition. We present a learning algorithm based on the methods of value iteration and upper confidence bound. We derive an upper bound on the worst case expected regret for the proposed algorithm and establish a worst case lower bound with both bounds of the order of square root on the number of episodes. Finally, we conduct simulation experiments to illustrate the performance of our algorithm. Funding: X. Gao is supported by the Hong Kong Research Grant Council [Grants 14201421, 14212522, 14200123]. X. Zhou gratefully acknowledges financial support through the Nie Center for Intelligent Asset Management at Columbia. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Mathematics of Operations Research (INFORMS) is the property of INFORMS: Institute for Operations Research & the Management Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=bsx&AN=191729069
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1287/moor.2022.0283
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
        StartPage: 333
    Subjects:
      – SubjectFull: Markov processes
        Type: general
      – SubjectFull: Dynamic programming
        Type: general
      – SubjectFull: Simulation methods & models
        Type: general
      – SubjectFull: Reinforcement learning
        Type: general
      – SubjectFull: Confidence intervals
        Type: general
    Titles:
      – TitleFull: Square-Root Regret Bounds for Continuous-Time Episodic Markov Decision Processes.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Gao, Xuefeng
      – PersonEntity:
          Name:
            NameFull: Zhou, Xunyu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Text: Feb2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 0364765X
          Numbering:
            – Type: volume
              Value: 51
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Mathematics of Operations Research (INFORMS)
              Type: main
ResultId 1