Academic Journal
Προβολή σε EDS

From natural language to executable filter code: LLM-Assisted behavioural customisation in trigger–Action Platforms.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: From natural language to executable filter code: LLM-Assisted behavioural customisation in trigger–Action Platforms.
Συγγραφείς: Cimino, Gaetano1 (AUTHOR) gcimino@unisa.it, De Santis, Laura1 (AUTHOR), Deufemia, Vincenzo1 (AUTHOR)
Πηγή: Behaviour & Information Technology. Sep2026, Vol. 45 Issue 15, p3515-3544. 30p.
Θεματικοί όροι: *Internet searching, *Generative artificial intelligence, *Computer software, *Task performance, *Data analysis, *Internet, *Natural language processing, *Descriptive statistics, *Workflow, *Statistics, *Automation, Cronbach's alpha, Research evaluation, Privacy, Kruskal-Wallis Test, Mann Whitney U Test, Surveys, Cataloging, Software architecture, User-centered system design, Confidence intervals, Rules, Medical ethics
Περίληψη: Trigger–Action Platforms (TAPs) are a widely adopted class of End-User Development systems that enable non-programmer users to automate interactions among online services and IoT devices through simple IF–THEN rules. Although advanced mechanisms such as conditional execution and data transformation increase expressiveness, they typically require programming expertise and remain inaccessible to many users, leaving a persistent gap between users' automation needs and what template-based interfaces support. This paper presents an LLM-assisted approach for behavioural customisation in TAPs, enabling the generation of executable filter code from natural language intents. The approach grounds generation in platform-provided catalogs of admissible variables and methods, guiding models toward syntactically valid and deployable solutions, and supports an interactive workflow in which users iteratively refine automation behaviour through natural language. We evaluate the approach through a mixed-method study combining automated experiments and a controlled user study. The automated evaluation, conducted on 351 real-world rules, assesses platform compliance and behavioural correctness using surface-level metrics and a semantics-aware analysis grounded in the Trigger–Action execution model. The best-performing configuration achieves Syntax rates exceeding.97, API scores around.70, and CodeBERTScore approaching.79. The user study, involving 85 participants and 510 tasks, indicates that LLM assistance, in a controlled study setting, helps users converge toward correct automations, with success rates above 95%, good perceived usability (SUS M = 69.9), manageable cognitive workload (NASA-TLX M = 24.9), and moderately high trust in the generated solutions (TOAST M = 5.1). [ABSTRACT FROM AUTHOR]
Copyright of Behaviour & Information Technology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Βάση Δεδομένων: Business Source Index
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:bsx&genre=article&issn=0144929X&ISBN=&volume=45&issue=15&date=20260915&spage=3515&pages=3515-3544&title=Behaviour & Information Technology&atitle=From%20natural%20language%20to%20executable%20filter%20code%3A%20LLM-Assisted%20behavioural%20customisation%20in%20trigger%E2%80%93Action%20Platforms.&aulast=Cimino%2C%20Gaetano&id=DOI:10.1080/0144929X.2026.2711012
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: bsx
DbLabel: Business Source Index
An: 197548463
RelevancyScore: 1435
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1435
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: From natural language to executable filter code: LLM-Assisted behavioural customisation in trigger–Action Platforms.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Cimino%2C+Gaetano%22">Cimino, Gaetano</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> gcimino@unisa.it</i><br /><searchLink fieldCode="AR" term="%22De+Santis%2C+Laura%22">De Santis, Laura</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Deufemia%2C+Vincenzo%22">Deufemia, Vincenzo</searchLink><relatesTo>1</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Behaviour+%26+Information+Technology%22">Behaviour & Information Technology</searchLink>. Sep2026, Vol. 45 Issue 15, p3515-3544. 30p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Internet+searching%22">Internet searching</searchLink><br />*<searchLink fieldCode="DE" term="%22Generative+artificial+intelligence%22">Generative artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Computer+software%22">Computer software</searchLink><br />*<searchLink fieldCode="DE" term="%22Task+performance%22">Task performance</searchLink><br />*<searchLink fieldCode="DE" term="%22Data+analysis%22">Data analysis</searchLink><br />*<searchLink fieldCode="DE" term="%22Internet%22">Internet</searchLink><br />*<searchLink fieldCode="DE" term="%22Natural+language+processing%22">Natural language processing</searchLink><br />*<searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br />*<searchLink fieldCode="DE" term="%22Workflow%22">Workflow</searchLink><br />*<searchLink fieldCode="DE" term="%22Statistics%22">Statistics</searchLink><br />*<searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Cronbach's+alpha%22">Cronbach's alpha</searchLink><br /><searchLink fieldCode="DE" term="%22Research+evaluation%22">Research evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Privacy%22">Privacy</searchLink><br /><searchLink fieldCode="DE" term="%22Kruskal-Wallis+Test%22">Kruskal-Wallis Test</searchLink><br /><searchLink fieldCode="DE" term="%22Mann+Whitney+U+Test%22">Mann Whitney U Test</searchLink><br /><searchLink fieldCode="DE" term="%22Surveys%22">Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Cataloging%22">Cataloging</searchLink><br /><searchLink fieldCode="DE" term="%22Software+architecture%22">Software architecture</searchLink><br /><searchLink fieldCode="DE" term="%22User-centered+system+design%22">User-centered system design</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+intervals%22">Confidence intervals</searchLink><br /><searchLink fieldCode="DE" term="%22Rules%22">Rules</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+ethics%22">Medical ethics</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Trigger–Action Platforms (TAPs) are a widely adopted class of End-User Development systems that enable non-programmer users to automate interactions among online services and IoT devices through simple IF–THEN rules. Although advanced mechanisms such as conditional execution and data transformation increase expressiveness, they typically require programming expertise and remain inaccessible to many users, leaving a persistent gap between users' automation needs and what template-based interfaces support. This paper presents an LLM-assisted approach for behavioural customisation in TAPs, enabling the generation of executable filter code from natural language intents. The approach grounds generation in platform-provided catalogs of admissible variables and methods, guiding models toward syntactically valid and deployable solutions, and supports an interactive workflow in which users iteratively refine automation behaviour through natural language. We evaluate the approach through a mixed-method study combining automated experiments and a controlled user study. The automated evaluation, conducted on 351 real-world rules, assesses platform compliance and behavioural correctness using surface-level metrics and a semantics-aware analysis grounded in the Trigger–Action execution model. The best-performing configuration achieves Syntax rates exceeding.97, API scores around.70, and CodeBERTScore approaching.79. The user study, involving 85 participants and 510 tasks, indicates that LLM assistance, in a controlled study setting, helps users converge toward correct automations, with success rates above 95%, good perceived usability (SUS M = 69.9), manageable cognitive workload (NASA-TLX M = 24.9), and moderately high trust in the generated solutions (TOAST M = 5.1). [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Behaviour & Information Technology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=bsx&AN=197548463
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/0144929X.2026.2711012
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 30
        StartPage: 3515
    Subjects:
      – SubjectFull: Internet searching
        Type: general
      – SubjectFull: Generative artificial intelligence
        Type: general
      – SubjectFull: Computer software
        Type: general
      – SubjectFull: Task performance
        Type: general
      – SubjectFull: Data analysis
        Type: general
      – SubjectFull: Internet
        Type: general
      – SubjectFull: Natural language processing
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Workflow
        Type: general
      – SubjectFull: Statistics
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Cronbach's alpha
        Type: general
      – SubjectFull: Research evaluation
        Type: general
      – SubjectFull: Privacy
        Type: general
      – SubjectFull: Kruskal-Wallis Test
        Type: general
      – SubjectFull: Mann Whitney U Test
        Type: general
      – SubjectFull: Surveys
        Type: general
      – SubjectFull: Cataloging
        Type: general
      – SubjectFull: Software architecture
        Type: general
      – SubjectFull: User-centered system design
        Type: general
      – SubjectFull: Confidence intervals
        Type: general
      – SubjectFull: Rules
        Type: general
      – SubjectFull: Medical ethics
        Type: general
    Titles:
      – TitleFull: From natural language to executable filter code: LLM-Assisted behavioural customisation in trigger–Action Platforms.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Cimino, Gaetano
      – PersonEntity:
          Name:
            NameFull: De Santis, Laura
      – PersonEntity:
          Name:
            NameFull: Deufemia, Vincenzo
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 15
              M: 09
              Text: Sep2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 0144929X
          Numbering:
            – Type: volume
              Value: 45
            – Type: issue
              Value: 15
          Titles:
            – TitleFull: Behaviour & Information Technology
              Type: main
ResultId 1