Academic Journal
View in EDS

Vanilla Bayesian Optimization Performs Great in High Dimensions

Bibliographic Details
Title: Vanilla Bayesian Optimization Performs Great in High Dimensions
Authors: Hvarfner, Carl, Hellsten, Erik O., Nardi, Luigi
Contributors: Lund University, Profile areas and other strong research environments, Strategic research areas (SRA), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Lunds universitet, Profilområden och andra starka forskningsmiljöer, Strategiska forskningsområden (SFO), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Originator, Lund University, Faculty of Engineering, LTH, Departments at LTH, Department of Computer Science, Robotics and Semantic Systems, Lunds universitet, Lunds Tekniska Högskola, Institutioner vid LTH, Institutionen för datavetenskap, Robotik och Semantiska System, Originator, Lund University, Faculty of Science, Centre for Mathematical Sciences, Mathematics (Faculty of Engineering), Lunds universitet, Naturvetenskapliga fakulteten, Matematikcentrum, Matematik LTH, Originator, Lund University, Faculty of Engineering, LTH, LTH Profile areas, LTH Profile Area: AI and Digitalization, Lunds universitet, Lunds Tekniska Högskola, LTH profilområden, LTH profilområde: AI och digitalisering, Originator
Source: Proceedings of Machine Learning Research Bayesian optimization across the spectrum of knowledge. 235:20793-20817
Subject Terms: Natural Sciences, Computer and Information Sciences, Computer graphics and computer vision, Naturvetenskap, Data- och informationsvetenskap (Datateknik), Datorgrafik och datorseende, Mathematical Sciences, Computational Mathematics, Matematik, Beräkningsmatematik
Description: High-dimensional problems have long been considered the Achilles' heel of Bayesian optimization. Spurred by the curse of dimensionality, a large collection of algorithms aim to make it more performant in this setting, commonly by imposing various simplifying assumptions on the objective. In this paper, we identify the degeneracies that make vanilla Bayesian optimization poorly suited to high-dimensional tasks, and further show how existing algorithms address these degeneracies through the lens of lowering the model complexity. Moreover, we propose an enhancement to the prior assumptions that are typical to vanilla Bayesian optimization, which reduces the complexity to manageable levels without imposing structural restrictions on the objective. Our modification - a simple scaling of the Gaussian process lengthscale prior with the dimensionality - reveals that standard Bayesian optimization works drastically better than previously thought in high dimensions, clearly outperforming existing state-of-the-art algorithms on multiple commonly considered real-world high-dimensional tasks.
Access URL: https://proceedings.mlr.press/v235/hussain24a.html
Database: SwePub
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://proceedings.mlr.press/v235/hussain24a.html#
    Name: EDS - SwePub (ns324271)
    Category: fullText
    Text: View record in SwePub
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:edsswe&genre=article&issn=26403498&ISBN=&volume=235&issue=&date=20240101&spage=20793&pages=20793-20817&title=Proceedings of Machine Learning Research Bayesian optimization across the spectrum of knowledge&atitle=Vanilla%20Bayesian%20Optimization%20Performs%20Great%20in%20High%20Dimensions&aulast=Hvarfner%2C%20Carl&id=DOI:
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: edsswe
DbLabel: SwePub
An: edsswe.oai.portal.research.lu.se.publications.4d9e7bc3.2aab.4c5d.a341.67c40b8e20d3
RelevancyScore: 1064
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1064.41540527344
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Vanilla Bayesian Optimization Performs Great in High Dimensions
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hvarfner%2C+Carl%22">Hvarfner, Carl</searchLink><br /><searchLink fieldCode="AR" term="%22Hellsten%2C+Erik+O%2E%22">Hellsten, Erik O.</searchLink><br /><searchLink fieldCode="AR" term="%22Nardi%2C+Luigi%22">Nardi, Luigi</searchLink>
– Name: Author
  Label: Contributors
  Group: Au
  Data: Lund University, Profile areas and other strong research environments, Strategic research areas (SRA), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Lunds universitet, Profilområden och andra starka forskningsmiljöer, Strategiska forskningsområden (SFO), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Originator<br />Lund University, Faculty of Engineering, LTH, Departments at LTH, Department of Computer Science, Robotics and Semantic Systems, Lunds universitet, Lunds Tekniska Högskola, Institutioner vid LTH, Institutionen för datavetenskap, Robotik och Semantiska System, Originator<br />Lund University, Faculty of Science, Centre for Mathematical Sciences, Mathematics (Faculty of Engineering), Lunds universitet, Naturvetenskapliga fakulteten, Matematikcentrum, Matematik LTH, Originator<br />Lund University, Faculty of Engineering, LTH, LTH Profile areas, LTH Profile Area: AI and Digitalization, Lunds universitet, Lunds Tekniska Högskola, LTH profilområden, LTH profilområde: AI och digitalisering, Originator
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <i>Proceedings of Machine Learning Research Bayesian optimization across the spectrum of knowledge</i>. 235:20793-20817
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Natural+Sciences%22">Natural Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+and+Information+Sciences%22">Computer and Information Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+graphics+and+computer+vision%22">Computer graphics and computer vision</searchLink><br /><searchLink fieldCode="DE" term="%22Naturvetenskap%22">Naturvetenskap</searchLink><br /><searchLink fieldCode="DE" term="%22Data-+och+informationsvetenskap+%28Datateknik%29%22">Data- och informationsvetenskap (Datateknik)</searchLink><br /><searchLink fieldCode="DE" term="%22Datorgrafik+och+datorseende%22">Datorgrafik och datorseende</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+Sciences%22">Mathematical Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Mathematics%22">Computational Mathematics</searchLink><br /><searchLink fieldCode="DE" term="%22Matematik%22">Matematik</searchLink><br /><searchLink fieldCode="DE" term="%22Beräkningsmatematik%22">Beräkningsmatematik</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: High-dimensional problems have long been considered the Achilles' heel of Bayesian optimization. Spurred by the curse of dimensionality, a large collection of algorithms aim to make it more performant in this setting, commonly by imposing various simplifying assumptions on the objective. In this paper, we identify the degeneracies that make vanilla Bayesian optimization poorly suited to high-dimensional tasks, and further show how existing algorithms address these degeneracies through the lens of lowering the model complexity. Moreover, we propose an enhancement to the prior assumptions that are typical to vanilla Bayesian optimization, which reduces the complexity to manageable levels without imposing structural restrictions on the objective. Our modification - a simple scaling of the Gaussian process lengthscale prior with the dimensionality - reveals that standard Bayesian optimization works drastically better than previously thought in high dimensions, clearly outperforming existing state-of-the-art algorithms on multiple commonly considered real-world high-dimensional tasks.
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://proceedings.mlr.press/v235/hussain24a.html" linkWindow="_blank">https://proceedings.mlr.press/v235/hussain24a.html</link>
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsswe&AN=edsswe.oai.portal.research.lu.se.publications.4d9e7bc3.2aab.4c5d.a341.67c40b8e20d3
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
        StartPage: 20793
    Subjects:
      – SubjectFull: Natural Sciences
        Type: general
      – SubjectFull: Computer and Information Sciences
        Type: general
      – SubjectFull: Computer graphics and computer vision
        Type: general
      – SubjectFull: Naturvetenskap
        Type: general
      – SubjectFull: Data- och informationsvetenskap (Datateknik)
        Type: general
      – SubjectFull: Datorgrafik och datorseende
        Type: general
      – SubjectFull: Mathematical Sciences
        Type: general
      – SubjectFull: Computational Mathematics
        Type: general
      – SubjectFull: Matematik
        Type: general
      – SubjectFull: Beräkningsmatematik
        Type: general
    Titles:
      – TitleFull: Vanilla Bayesian Optimization Performs Great in High Dimensions
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hvarfner, Carl
      – PersonEntity:
          Name:
            NameFull: Hellsten, Erik O.
      – PersonEntity:
          Name:
            NameFull: Nardi, Luigi
      – PersonEntity:
          Name:
            NameFull: Lund University, Profile areas and other strong research environments, Strategic research areas (SRA), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Lunds universitet, Profilområden och andra starka forskningsmiljöer, Strategiska forskningsområden (SFO), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Originator
      – PersonEntity:
          Name:
            NameFull: Lund University, Faculty of Engineering, LTH, Departments at LTH, Department of Computer Science, Robotics and Semantic Systems, Lunds universitet, Lunds Tekniska Högskola, Institutioner vid LTH, Institutionen för datavetenskap, Robotik och Semantiska System, Originator
      – PersonEntity:
          Name:
            NameFull: Lund University, Faculty of Science, Centre for Mathematical Sciences, Mathematics (Faculty of Engineering), Lunds universitet, Naturvetenskapliga fakulteten, Matematikcentrum, Matematik LTH, Originator
      – PersonEntity:
          Name:
            NameFull: Lund University, Faculty of Engineering, LTH, LTH Profile areas, LTH Profile Area: AI and Digitalization, Lunds universitet, Lunds Tekniska Högskola, LTH profilområden, LTH profilområde: AI och digitalisering, Originator
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 26403498
            – Type: issn-locals
              Value: SWEPUB_FREE
            – Type: issn-locals
              Value: LU_SWEPUB
          Numbering:
            – Type: volume
              Value: 235
          Titles:
            – TitleFull: Proceedings of Machine Learning Research Bayesian optimization across the spectrum of knowledge
              Type: main
ResultId 1