Academic Journal
Προβολή σε EDS

Relaxed Equilibria for Time-Inconsistent Markov Decision Processes.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Relaxed Equilibria for Time-Inconsistent Markov Decision Processes.
Συγγραφείς: Bayraktar, Erhan1 (AUTHOR) erhan@umich.edu, Huang, Yu-Jui2 (AUTHOR) yujui.huang@colorado.edu, Wang, Zhenhua3 (AUTHOR) zhenhuaw@sdu.edu.cn, Zhou, Zhou4 (AUTHOR) zhou.zhou@sydney.edu.au
Πηγή: Mathematics of Operations Research (INFORMS). Nov2025, Vol. 50 Issue 4, p2666-2687. 22p.
Θεματικοί όροι: *Markov processes, *Delay discounting (Psychology), *Game theory, Equilibrium, Time-varying systems, Mathematical regularization
Περίληψη: This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent's current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case. [ABSTRACT FROM AUTHOR]
Copyright of Mathematics of Operations Research (INFORMS) is the property of INFORMS: Institute for Operations Research & the Management Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Βάση Δεδομένων: Business Source Index
FullText Text:
  Availability: 0
CustomLinks:
  – Url: http://www.jstor.org/openurl?issn=0364765X&date=2025&volume=50&issue=4&spage=2666
    Name: JSTOR (all content) (s7799221)
    Category: fullText
    Text:
    Icon: https://imageserver.ebscohost.com/branding/jstor/iconJSTOR.gif
    MouseOverText: Full Text from JSTOR
Header DbId: bsx
DbLabel: Business Source Index
An: 189423739
RelevancyScore: 1451
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1451.22045898438
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Relaxed Equilibria for Time-Inconsistent Markov Decision Processes.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Bayraktar%2C+Erhan%22">Bayraktar, Erhan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> erhan@umich.edu</i><br /><searchLink fieldCode="AR" term="%22Huang%2C+Yu-Jui%22">Huang, Yu-Jui</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> yujui.huang@colorado.edu</i><br /><searchLink fieldCode="AR" term="%22Wang%2C+Zhenhua%22">Wang, Zhenhua</searchLink><relatesTo>3</relatesTo> (AUTHOR)<i> zhenhuaw@sdu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Zhou%2C+Zhou%22">Zhou, Zhou</searchLink><relatesTo>4</relatesTo> (AUTHOR)<i> zhou.zhou@sydney.edu.au</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Mathematics+of+Operations+Research+%28INFORMS%29%22">Mathematics of Operations Research (INFORMS)</searchLink>. Nov2025, Vol. 50 Issue 4, p2666-2687. 22p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Markov+processes%22">Markov processes</searchLink><br />*<searchLink fieldCode="DE" term="%22Delay+discounting+%28Psychology%29%22">Delay discounting (Psychology)</searchLink><br />*<searchLink fieldCode="DE" term="%22Game+theory%22">Game theory</searchLink><br /><searchLink fieldCode="DE" term="%22Equilibrium%22">Equilibrium</searchLink><br /><searchLink fieldCode="DE" term="%22Time-varying+systems%22">Time-varying systems</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+regularization%22">Mathematical regularization</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent's current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Mathematics of Operations Research (INFORMS) is the property of INFORMS: Institute for Operations Research & the Management Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=bsx&AN=189423739
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1287/moor.2023.0209
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 22
        StartPage: 2666
    Subjects:
      – SubjectFull: Markov processes
        Type: general
      – SubjectFull: Delay discounting (Psychology)
        Type: general
      – SubjectFull: Game theory
        Type: general
      – SubjectFull: Equilibrium
        Type: general
      – SubjectFull: Time-varying systems
        Type: general
      – SubjectFull: Mathematical regularization
        Type: general
    Titles:
      – TitleFull: Relaxed Equilibria for Time-Inconsistent Markov Decision Processes.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Bayraktar, Erhan
      – PersonEntity:
          Name:
            NameFull: Huang, Yu-Jui
      – PersonEntity:
          Name:
            NameFull: Wang, Zhenhua
      – PersonEntity:
          Name:
            NameFull: Zhou, Zhou
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Text: Nov2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0364765X
          Numbering:
            – Type: volume
              Value: 50
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Mathematics of Operations Research (INFORMS)
              Type: main
ResultId 1