Academic Journal
View in EDS

Effects of a Safety User Interface Bundle on Verification Intentions in Generative AI Chat Use Among Older Chinese Adults: Randomized Vignette Survey.

Bibliographic Details
Title: Effects of a Safety User Interface Bundle on Verification Intentions in Generative AI Chat Use Among Older Chinese Adults: Randomized Vignette Survey.
Authors: Yu J; Faculty of Science, University of Auckland, Auckland, New Zealand., Chen J; Medical Services Management Department, Peking University People's Hospital (PKUPH), Beijing, China., Ren A; Healthcare & Education Research Center, Chengdu Gongyun Education & Management Research Institute, Chengdu, China., Duan H; School of Public Administration and Policy, Renmin University of China, Beijing, China., Meng H; Department of Economics and Management, Sichuan University of Architectural Technology, Chengdu, China., Gao Z; Department of Human Resource Management, Beijing Geriatric Hospital, 118 Wenquan Road, Haidian District, Beijing, 100095, China, 86 15201400966.
Source: Journal of medical Internet research [J Med Internet Res] 2026 Aug 14; Vol. 28, pp. e94140. Date of Electronic Publication: 2026 Aug 14.
Publication Type: Journal Article; Randomized Controlled Trial
Language: English
Journal Info: Publisher: JMIR Publications Country of Publication: Canada NLM ID: 100959882 Publication Model: Electronic Cited Medium: Internet ISSN: 1438-8871 (Electronic) Linking ISSN: 14388871 NLM ISO Abbreviation: J Med Internet Res Subsets: MEDLINE
Imprint Name(s): Publication: <2011- > : Toronto : JMIR Publications
Original Publication: [Pittsburgh, PA? : s.n., 1999-
MeSH Terms: Generative Artificial Intelligence* , Intention* , Safety* , User-Computer Interface*, Aged ; Female ; Humans ; Male ; Middle Aged ; China ; Cross-Sectional Studies ; Surveys and Questionnaires ; Trust ; East Asian People
Abstract: Background: Generative AI chat systems are increasingly used for everyday information seeking, but plausible errors and omissions can mislead users when outputs are accepted without scrutiny. Interface-level safety cues may help users calibrate trust and engage in verification; yet, evidence in older Chinese adults remains limited.
Objective: This study aimed to test whether adding a safety user interface (UI) bundle to a generative AI chat interface increases verification intention among older Chinese adults and to examine selected secondary outcomes, including reliance intention, trust calibration, perceived trustworthiness, comprehension, usability/readability, cognitive load, and a behavioral proxy of verification.
Methods: We conducted a cross-sectional survey with an embedded randomized UI vignette experiment between May 22, 2025, and September 3, 2025. Chinese adults aged ≥60 years were recruited through community sites, outpatient clinic waiting areas, and WeChat (Tencent Holdings Ltd) groups, and randomized 1:1 to view screenshots of a baseline chat UI or a safety UI bundle containing generic source-label cues, and an uncertainty and verification nudge. Each participant completed 2 scenarios (service/travel decision and general well-being related to sleep/fatigue), followed by measures of verification intention (primary), reliance intention, trust calibration index, comprehension (0-8), perceived trustworthiness, usability/readability, cognitive load (0-10), manipulation checks, and a behavioral proxy (expanding optional "source information"). Analyses used intention-to-treat regression models with covariate adjustment.
Results: Of 214 consenting respondents who started the survey, 200 were included in the analysis (100 per arm). The safety UI bundle increased verification intention (mean 4.72, SD 0.63 vs 4.41, SD 0.59 on a 7-point scale; adjusted β=0.293, 95% CI 0.128-0.457; P<.001). Reliance intention did not increase (mean 4.97, SD 0.54 vs 5.03, SD 0.58; adjusted β=-0.105, 95% CI -0.239 to 0.029; P=.13). Trust calibration improved (trust calibration index: mean -0.29, SD 1.43 vs 0.29, SD 1.43; adjusted β=-0.567, 95% CI -1.005 to -0.129; P=.01). Expansion of optional source information was numerically higher, although the adjusted CI included the null (42% vs 27%; adjusted odds ratio [OR]=1.76, 95% CI 0.95-3.27; P=.07). Comprehension remained high and similar across arms (mean 6.33, SD 1.14 vs 6.32, SD 1.08; adjusted β=-0.132, 95% CI -0.428 to 0.163; P=.38). Perceived trustworthiness was modestly lower in the Safety UI arm (mean 5.20, SD 0.61 vs 5.39, SD 0.66; adjusted β=-0.199, 95% CI -0.382 to -0.016; P=.03). Usability/readability was unchanged, and cognitive load did not increase. Manipulation checks indicated higher cue recognition in the Safety UI arm.
Conclusions: In a randomized static-vignette survey of older Chinese adults, a brief safety UI bundle was associated with higher verification intention and a trust calibration index consistent with lower overreliance risk, without detectable reductions in comprehension or usability/readability. Because the intervention was tested as a bundle using screenshots and generic source labels, findings should be interpreted as evidence for a practical interface-level strategy rather than proof that any single cue caused the observed effects.
(© Jun'an Yu, Jun Chen, Anjie Ren, Hui Duan, Hua Meng, Zhuo Gao. Originally published in the Journal of Medical Internet Research (https://www.jmir.org).)
References: Patterns (N Y). 2022 Feb 24;3(4):100455. (PMID: 35465233)
Front Public Health. 2025 Sep 04;13:1637270. (PMID: 40977766)
JMIR Aging. 2023 May 16;6:e44564. (PMID: 37191976)
BMC Psychol. 2024 May 8;12(1):255. (PMID: 38720382)
Front Public Health. 2024 Nov 19;12:1435329. (PMID: 39628811)
Front Psychol. 2024 Apr 17;15:1382693. (PMID: 38694439)
Hum Factors. 2025 Oct;67(10):1062-1083. (PMID: 40104968)
J Med Internet Res. 2023 Jun 14;25:e47184. (PMID: 37314848)
PeerJ Comput Sci. 2025 Mar 27;11:e2773. (PMID: 40567759)
J Am Med Inform Assoc. 2024 Nov 1;31(11):2730-2739. (PMID: 39325508)
Hum Factors. 2024 Jan;66(1):126-144. (PMID: 35344676)
World Wide Web. 2021;24(5):1857-1884. (PMID: 34366701)
Contributed Indexing: Keywords: China; generative AI; human-computer interaction; large language models; older adults; randomized experiment; safety cues; trust calibration; user interface; verification; vignette survey
Entry Date(s): Date Created: 20260814 Date Completed: 20260814 Latest Revision: 20260817
Update Code: 20260817
PubMed Central ID: PMC13475784
DOI: 10.2196/94140
PMID: 42600074
Database: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=14388871&ISBN=&volume=28&issue=&date=20260814&spage=e94140&pages=e94140&title=Journal of medical Internet research&atitle=Effects%20of%20a%20Safety%20User%20Interface%20Bundle%20on%20Verification%20Intentions%20in%20Generative%20AI%20Chat%20Use%20Among%20Older%20Chinese%20Adults%3A%20Randomized%20Vignette%20Survey.&aulast=Yu%20J&id=DOI:10.2196/94140
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 42600074
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Effects of a Safety User Interface Bundle on Verification Intentions in Generative AI Chat Use Among Older Chinese Adults: Randomized Vignette Survey.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Yu+J%22&quot;&gt;Yu J&lt;/searchLink&gt;; Faculty of Science, University of Auckland, Auckland, New Zealand.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Chen+J%22&quot;&gt;Chen J&lt;/searchLink&gt;; Medical Services Management Department, Peking University People&#39;s Hospital (PKUPH), Beijing, China.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Ren+A%22&quot;&gt;Ren A&lt;/searchLink&gt;; Healthcare &amp; Education Research Center, Chengdu Gongyun Education &amp; Management Research Institute, Chengdu, China.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Duan+H%22&quot;&gt;Duan H&lt;/searchLink&gt;; School of Public Administration and Policy, Renmin University of China, Beijing, China.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Meng+H%22&quot;&gt;Meng H&lt;/searchLink&gt;; Department of Economics and Management, Sichuan University of Architectural Technology, Chengdu, China.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Gao+Z%22&quot;&gt;Gao Z&lt;/searchLink&gt;; Department of Human Resource Management, Beijing Geriatric Hospital, 118 Wenquan Road, Haidian District, Beijing, 100095, China, 86 15201400966.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%22100959882%22&quot;&gt;Journal of medical Internet research&lt;/searchLink&gt; [J Med Internet Res] 2026 Aug 14; Vol. 28, pp. e94140. &lt;i&gt;Date of Electronic Publication: &lt;/i&gt;2026 Aug 14.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article; Randomized Controlled Trial
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: &lt;i&gt;Publisher: &lt;/i&gt;&lt;searchLink fieldCode=&quot;PB&quot; term=&quot;%22JMIR+Publications%22&quot;&gt;JMIR Publications &lt;/searchLink&gt;&lt;i&gt;Country of Publication: &lt;/i&gt;Canada &lt;i&gt;NLM ID: &lt;/i&gt;100959882 &lt;i&gt;Publication Model: &lt;/i&gt;Electronic &lt;i&gt;Cited Medium: &lt;/i&gt;Internet &lt;i&gt;ISSN: &lt;/i&gt;1438-8871 (Electronic) &lt;i&gt;Linking ISSN: &lt;/i&gt;&lt;searchLink fieldCode=&quot;IS&quot; term=&quot;%2214388871%22&quot;&gt;14388871 &lt;/searchLink&gt;&lt;i&gt;NLM ISO Abbreviation: &lt;/i&gt;J Med Internet Res &lt;i&gt;Subsets: &lt;/i&gt;MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: &lt;i&gt;Publication&lt;/i&gt;: &lt;2011- &gt; : Toronto : JMIR Publications&lt;br /&gt;&lt;i&gt;Original Publication&lt;/i&gt;: [Pittsburgh, PA? : s.n., 1999-
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Generative+Artificial+Intelligence%22&quot;&gt;Generative Artificial Intelligence*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Intention%22&quot;&gt;Intention*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Safety%22&quot;&gt;Safety*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22User-Computer+Interface%22&quot;&gt;User-Computer Interface*&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Aged%22&quot;&gt;Aged&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Female%22&quot;&gt;Female&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Humans%22&quot;&gt;Humans&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Male%22&quot;&gt;Male&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Middle+Aged%22&quot;&gt;Middle Aged&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22China%22&quot;&gt;China&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Cross-Sectional+Studies%22&quot;&gt;Cross-Sectional Studies&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Surveys+and+Questionnaires%22&quot;&gt;Surveys and Questionnaires&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Trust%22&quot;&gt;Trust&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22East+Asian+People%22&quot;&gt;East Asian People&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Generative AI chat systems are increasingly used for everyday information seeking, but plausible errors and omissions can mislead users when outputs are accepted without scrutiny. Interface-level safety cues may help users calibrate trust and engage in verification; yet, evidence in older Chinese adults remains limited.&lt;br /&gt;Objective: This study aimed to test whether adding a safety user interface (UI) bundle to a generative AI chat interface increases verification intention among older Chinese adults and to examine selected secondary outcomes, including reliance intention, trust calibration, perceived trustworthiness, comprehension, usability/readability, cognitive load, and a behavioral proxy of verification.&lt;br /&gt;Methods: We conducted a cross-sectional survey with an embedded randomized UI vignette experiment between May 22, 2025, and September 3, 2025. Chinese adults aged ≥60 years were recruited through community sites, outpatient clinic waiting areas, and WeChat (Tencent Holdings Ltd) groups, and randomized 1:1 to view screenshots of a baseline chat UI or a safety UI bundle containing generic source-label cues, and an uncertainty and verification nudge. Each participant completed 2 scenarios (service/travel decision and general well-being related to sleep/fatigue), followed by measures of verification intention (primary), reliance intention, trust calibration index, comprehension (0-8), perceived trustworthiness, usability/readability, cognitive load (0-10), manipulation checks, and a behavioral proxy (expanding optional &quot;source information&quot;). Analyses used intention-to-treat regression models with covariate adjustment.&lt;br /&gt;Results: Of 214 consenting respondents who started the survey, 200 were included in the analysis (100 per arm). The safety UI bundle increased verification intention (mean 4.72, SD 0.63 vs 4.41, SD 0.59 on a 7-point scale; adjusted β=0.293, 95% CI 0.128-0.457; P&amp;lt;.001). Reliance intention did not increase (mean 4.97, SD 0.54 vs 5.03, SD 0.58; adjusted β=-0.105, 95% CI -0.239 to 0.029; P=.13). Trust calibration improved (trust calibration index: mean -0.29, SD 1.43 vs 0.29, SD 1.43; adjusted β=-0.567, 95% CI -1.005 to -0.129; P=.01). Expansion of optional source information was numerically higher, although the adjusted CI included the null (42% vs 27%; adjusted odds ratio [OR]=1.76, 95% CI 0.95-3.27; P=.07). Comprehension remained high and similar across arms (mean 6.33, SD 1.14 vs 6.32, SD 1.08; adjusted β=-0.132, 95% CI -0.428 to 0.163; P=.38). Perceived trustworthiness was modestly lower in the Safety UI arm (mean 5.20, SD 0.61 vs 5.39, SD 0.66; adjusted β=-0.199, 95% CI -0.382 to -0.016; P=.03). Usability/readability was unchanged, and cognitive load did not increase. Manipulation checks indicated higher cue recognition in the Safety UI arm.&lt;br /&gt;Conclusions: In a randomized static-vignette survey of older Chinese adults, a brief safety UI bundle was associated with higher verification intention and a trust calibration index consistent with lower overreliance risk, without detectable reductions in comprehension or usability/readability. Because the intervention was tested as a bundle using screenshots and generic source labels, findings should be interpreted as evidence for a practical interface-level strategy rather than proof that any single cue caused the observed effects.&lt;br /&gt; (&#169; Jun&#39;an Yu, Jun Chen, Anjie Ren, Hui Duan, Hua Meng, Zhuo Gao. Originally published in the Journal of Medical Internet Research (https://www.jmir.org).)
– Name: Ref
  Label: References
  Group: RefInfo
  Data: Patterns (N Y). 2022 Feb 24;3(4):100455. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2235465233%22&quot;&gt;35465233)&lt;/searchLink&gt;&lt;br /&gt;Front Public Health. 2025 Sep 04;13:1637270. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2240977766%22&quot;&gt;40977766)&lt;/searchLink&gt;&lt;br /&gt;JMIR Aging. 2023 May 16;6:e44564. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2237191976%22&quot;&gt;37191976)&lt;/searchLink&gt;&lt;br /&gt;BMC Psychol. 2024 May 8;12(1):255. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2238720382%22&quot;&gt;38720382)&lt;/searchLink&gt;&lt;br /&gt;Front Public Health. 2024 Nov 19;12:1435329. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2239628811%22&quot;&gt;39628811)&lt;/searchLink&gt;&lt;br /&gt;Front Psychol. 2024 Apr 17;15:1382693. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2238694439%22&quot;&gt;38694439)&lt;/searchLink&gt;&lt;br /&gt;Hum Factors. 2025 Oct;67(10):1062-1083. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2240104968%22&quot;&gt;40104968)&lt;/searchLink&gt;&lt;br /&gt;J Med Internet Res. 2023 Jun 14;25:e47184. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2237314848%22&quot;&gt;37314848)&lt;/searchLink&gt;&lt;br /&gt;PeerJ Comput Sci. 2025 Mar 27;11:e2773. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2240567759%22&quot;&gt;40567759)&lt;/searchLink&gt;&lt;br /&gt;J Am Med Inform Assoc. 2024 Nov 1;31(11):2730-2739. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2239325508%22&quot;&gt;39325508)&lt;/searchLink&gt;&lt;br /&gt;Hum Factors. 2024 Jan;66(1):126-144. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2235344676%22&quot;&gt;35344676)&lt;/searchLink&gt;&lt;br /&gt;World Wide Web. 2021;24(5):1857-1884. (PMID: &lt;searchLink fieldCode=&quot;PM&quot; term=&quot;%2234366701%22&quot;&gt;34366701)&lt;/searchLink&gt;
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: &lt;i&gt;Keywords: &lt;/i&gt;China; generative AI; human-computer interaction; large language models; older adults; randomized experiment; safety cues; trust calibration; user interface; verification; vignette survey
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: &lt;i&gt;Date Created: &lt;/i&gt;20260814 &lt;i&gt;Date Completed: &lt;/i&gt;20260814 &lt;i&gt;Latest Revision: &lt;/i&gt;20260817
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260817
– Name: PubmedCentralID
  Label: PubMed Central ID
  Group: ID
  Data: PMC13475784
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.2196/94140
– Name: AN
  Label: PMID
  Group: ID
  Data: 42600074
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42600074
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.2196/94140
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: e94140
    Subjects:
      – SubjectFull: Aged
        Type: general
      – SubjectFull: Female
        Type: general
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Male
        Type: general
      – SubjectFull: Middle Aged
        Type: general
      – SubjectFull: China
        Type: general
      – SubjectFull: Cross-Sectional Studies
        Type: general
      – SubjectFull: Surveys and Questionnaires
        Type: general
      – SubjectFull: Trust
        Type: general
      – SubjectFull: East Asian People
        Type: general
      – SubjectFull: Generative Artificial Intelligence
        Type: general
      – SubjectFull: Intention
        Type: general
      – SubjectFull: Safety
        Type: general
      – SubjectFull: User-Computer Interface
        Type: general
    Titles:
      – TitleFull: Effects of a Safety User Interface Bundle on Verification Intentions in Generative AI Chat Use Among Older Chinese Adults: Randomized Vignette Survey.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yu J
      – PersonEntity:
          Name:
            NameFull: Chen J
      – PersonEntity:
          Name:
            NameFull: Ren A
      – PersonEntity:
          Name:
            NameFull: Duan H
      – PersonEntity:
          Name:
            NameFull: Meng H
      – PersonEntity:
          Name:
            NameFull: Gao Z
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 14
              M: 08
              Text: 2026 Aug 14
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1438-8871
          Numbering:
            – Type: volume
              Value: 28
          Titles:
            – TitleFull: Journal of medical Internet research
              Type: main
ResultId 1