Academic Journal
View in EDS

Security Analysis of Large Language Models on API Misuse Programming Repair.

Bibliographic Details
Title: Security Analysis of Large Language Models on API Misuse Programming Repair.
Authors: Zhang, Rui, Qiao, Ziyue, Yu, Yong, Vocaturo, Eugenio
Source: International Journal of Intelligent Systems; 11/19/2024, Vol. 2024, p1-18, 18p
Subject Terms: Language models, Security systems software, Resource management, Empirical research, Awareness
Abstract: Application programming interface (API) misuse refers to misconceptions or carelessness in the anticipated usage of APIs, threatening the software system's security. Moreover, API misuses demonstrate significant concealment and are challenging to uncover. Recent advancements have explored enhanced LLMs in a variety of software engineering (SE) activities, such as code repair. Nonetheless, the security implications of using LLMs for these purposes remain underexplored, particularly concerning the issue of API misuse. In this paper, we present an empirical study to observe the bug‐fixing capabilities of LLMs in addressing API misuse related to monitoring resource management (MRM API misuse). Initially, we propose APImisRepair, a real‐world benchmark for repairing MRM API misuse, including buggy programs, corresponding fixed programs, and descriptions of API misuse. Subsequently, we assess the performance of several LLMs using the APImisRepair benchmark. Findings reveal the vulnerabilities of LLMs in repairing MRM API misuse and find several reasons, encompassing factors such as fault localization and a lack of awareness regarding API misuse. Additionally, we have insights on improving LLMs in terms of their ability to fix MRM API misuse and introduce a crafted approach, APImisAP. Experimental results demonstrate that APImisAP exhibits a certain degree of improvement in the security of LLMs. [ABSTRACT FROM AUTHOR]
Copyright of International Journal of Intelligent Systems is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Complementary Index
FullText Links:
  – Type: other
Text:
  Availability: 0
Header DbId: edb
DbLabel: Complementary Index
An: 180988191
RelevancyScore: 974
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 973.596313476563
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Security Analysis of Large Language Models on API Misuse Programming Repair.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Zhang%2C+Rui%22">Zhang, Rui</searchLink><br /><searchLink fieldCode="AR" term="%22Qiao%2C+Ziyue%22">Qiao, Ziyue</searchLink><br /><searchLink fieldCode="AR" term="%22Yu%2C+Yong%22">Yu, Yong</searchLink><br /><searchLink fieldCode="AR" term="%22Vocaturo%2C+Eugenio%22">Vocaturo, Eugenio</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: International Journal of Intelligent Systems; 11/19/2024, Vol. 2024, p1-18, 18p
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Security+systems+software%22">Security systems software</searchLink><br /><searchLink fieldCode="DE" term="%22Resource+management%22">Resource management</searchLink><br /><searchLink fieldCode="DE" term="%22Empirical+research%22">Empirical research</searchLink><br /><searchLink fieldCode="DE" term="%22Awareness%22">Awareness</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Application programming interface (API) misuse refers to misconceptions or carelessness in the anticipated usage of APIs, threatening the software system's security. Moreover, API misuses demonstrate significant concealment and are challenging to uncover. Recent advancements have explored enhanced LLMs in a variety of software engineering (SE) activities, such as code repair. Nonetheless, the security implications of using LLMs for these purposes remain underexplored, particularly concerning the issue of API misuse. In this paper, we present an empirical study to observe the bug‐fixing capabilities of LLMs in addressing API misuse related to monitoring resource management (MRM API misuse). Initially, we propose APImisRepair, a real‐world benchmark for repairing MRM API misuse, including buggy programs, corresponding fixed programs, and descriptions of API misuse. Subsequently, we assess the performance of several LLMs using the APImisRepair benchmark. Findings reveal the vulnerabilities of LLMs in repairing MRM API misuse and find several reasons, encompassing factors such as fault localization and a lack of awareness regarding API misuse. Additionally, we have insights on improving LLMs in terms of their ability to fix MRM API misuse and introduce a crafted approach, APImisAP. Experimental results demonstrate that APImisAP exhibits a certain degree of improvement in the security of LLMs. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label:
  Group: Ab
  Data: <i>Copyright of International Journal of Intelligent Systems is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edb&AN=180988191
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1155/2024/7135765
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 1
    Subjects:
      – SubjectFull: Language models
        Type: general
      – SubjectFull: Security systems software
        Type: general
      – SubjectFull: Resource management
        Type: general
      – SubjectFull: Empirical research
        Type: general
      – SubjectFull: Awareness
        Type: general
    Titles:
      – TitleFull: Security Analysis of Large Language Models on API Misuse Programming Repair.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Zhang, Rui
      – PersonEntity:
          Name:
            NameFull: Qiao, Ziyue
      – PersonEntity:
          Name:
            NameFull: Yu, Yong
      – PersonEntity:
          Name:
            NameFull: Vocaturo, Eugenio
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 19
              M: 11
              Text: 11/19/2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 08848173
          Numbering:
            – Type: volume
              Value: 2024
          Titles:
            – TitleFull: International Journal of Intelligent Systems
              Type: main
ResultId 1