Academic Journal
Προβολή σε EDS

Assessment of Large Language Models and Expert Clinicians in Grading Mandibular Third Molar Extraction Difficulty.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Assessment of Large Language Models and Expert Clinicians in Grading Mandibular Third Molar Extraction Difficulty.
Συγγραφείς: Yoon SM; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea., Yoo JH; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea., Kim T; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea., Kang JH; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea., Kim BC; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea. Electronic address: bck@wku.ac.kr.
Πηγή: International dental journal [Int Dent J] 2026 Oct; Vol. 76 (5), pp. 109736. Date of Electronic Publication: 2026 Jul 06.
Τύπος έκδοσης: Journal Article
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: Elsevier Country of Publication: England NLM ID: 0374714 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1875-595X (Electronic) Linking ISSN: 00206539 NLM ISO Abbreviation: Int Dent J Subsets: MEDLINE
Imprint Name(s): Publication: 2021- : London : Elsevier
Original Publication: London : FDI World Dental
Ιατρικοί όροι (MeSH): Molar, Third*/surgery , Molar, Third*/diagnostic imaging , Large Language Models* , Tooth Extraction*, Mandible/surgery ; Humans ; Retrospective Studies ; Cross-Sectional Studies ; Observer Variation ; Reproducibility of Results ; Radiography, Panoramic ; Female
Περίληψη: Background: The Pederson difficulty score (PDS) is widely used to assess mandibular third molar extraction difficulty, but is subject to interobserver variability because no universally accepted reference standard exists. This study investigates whether multimodal large language models (LLMs) demonstrate systematic grading tendencies comparable to those of human clinicians in the absence of a definitive standard.
Methods: This retrospective cross-sectional study included 100 panoramic radiographs. Two LLMs (GPT and Gemini) and two oral and maxillofacial surgeons independently graded extraction difficulty using the PDS. No reference standard or gold-standard rater was designated; all raters were treated as independent assessors. Agreement was assessed using quadratic-weighted kappa and intraclass correlation coefficient values. Systematic bias was analysed using Bland-Altman plots and ordinal logistic regression. The effects of prompt language and session protocol were also evaluated.
Results: The interexpert agreement was substantial (κ = 0.754). GPT demonstrated moderate agreement with experts (κ = 0.564-0.590), whereas Gemini exhibited fair-to-moderate agreement (κ = 0.356-0.461). Overall, the four-rater reliability was moderate (intraclass correlation coefficient = 0.553). Both LLMs displayed minimal systematic bias (GPT: -0.13; Gemini: -0.08), with no significant differences in grading tendencies compared with human experts (P > .05). However, sequential-session evaluation significantly reduced the scoring consistency for both models (GPT: P = .001; Gemini: P < .001), whereas prompt language had no significant effect on total PDS scores (GPT: P = .543; Gemini: P = .386). A component-level linguistic variation was observed only in GPT's assessment of the ramus relationship (P = .003).
Conclusion: The examined LLMs showed grading tendencies partly comparable to those of human clinicians, without evidence of systematic directional bias; however, agreement remained below interexpert levels and was sensitive to conversational context. These findings should not be interpreted as evidence of diagnostic accuracy or clinical reliability, as no reference standard was established.
Clinical Relevance: LLMs may provide supplementary grading information if used with independent-session protocols; however, their role in clinical decision-making requires further validation against defined outcome standards.
(Copyright © 2026 The Authors. Published by Elsevier Inc. All rights reserved.)
Competing Interests: Conflict of interest The authors declare that they have no competing interests.
Contributed Indexing: Keywords: Artificial intelligence; Decision making; Large language models; Third molar; Tooth extraction
Entry Date(s): Date Created: 20260706 Date Completed: 20260916 Latest Revision: 20260916
Update Code: 20260916
PubMed Central ID: PMC13351125
DOI: 10.1016/j.identj.2026.109736
PMID: 42407142
Βάση Δεδομένων: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://www.doi.org/10.1016/j.identj.2026.109736?
    Name: ScienceDirect (all content) (s7799221)
    Category: fullText
    Text: View record from ScienceDirect
    MouseOverText: View record from ScienceDirect
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=1875595X&ISBN=&volume=76&issue=5&date=20261001&spage=109736&pages=109736&title=International dental journal&atitle=Assessment%20of%20Large%20Language%20Models%20and%20Expert%20Clinicians%20in%20Grading%20Mandibular%20Third%20Molar%20Extraction%20Difficulty.&aulast=Yoon%20SM&id=DOI:10.1016/j.identj.2026.109736
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 42407142
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Assessment of Large Language Models and Expert Clinicians in Grading Mandibular Third Molar Extraction Difficulty.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Yoon+SM%22&quot;&gt;Yoon SM&lt;/searchLink&gt;; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Yoo+JH%22&quot;&gt;Yoo JH&lt;/searchLink&gt;; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Kim+T%22&quot;&gt;Kim T&lt;/searchLink&gt;; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Kang+JH%22&quot;&gt;Kang JH&lt;/searchLink&gt;; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Kim+BC%22&quot;&gt;Kim BC&lt;/searchLink&gt;; Department of Oral and Maxillofacial Surgery, Daejeon Dental Hospital, Wonkwang University College of Dentistry, Daejeon, Republic of Korea. Electronic address: bck@wku.ac.kr.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%220374714%22&quot;&gt;International dental journal&lt;/searchLink&gt; [Int Dent J] 2026 Oct; Vol. 76 (5), pp. 109736. &lt;i&gt;Date of Electronic Publication: &lt;/i&gt;2026 Jul 06.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: &lt;i&gt;Publisher: &lt;/i&gt;&lt;searchLink fieldCode=&quot;PB&quot; term=&quot;%22Elsevier%22&quot;&gt;Elsevier &lt;/searchLink&gt;&lt;i&gt;Country of Publication: &lt;/i&gt;England &lt;i&gt;NLM ID: &lt;/i&gt;0374714 &lt;i&gt;Publication Model: &lt;/i&gt;Print-Electronic &lt;i&gt;Cited Medium: &lt;/i&gt;Internet &lt;i&gt;ISSN: &lt;/i&gt;1875-595X (Electronic) &lt;i&gt;Linking ISSN: &lt;/i&gt;&lt;searchLink fieldCode=&quot;IS&quot; term=&quot;%2200206539%22&quot;&gt;00206539 &lt;/searchLink&gt;&lt;i&gt;NLM ISO Abbreviation: &lt;/i&gt;Int Dent J &lt;i&gt;Subsets: &lt;/i&gt;MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: &lt;i&gt;Publication&lt;/i&gt;: 2021- : London : Elsevier&lt;br /&gt;&lt;i&gt;Original Publication&lt;/i&gt;: London : FDI World Dental
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Molar%2C+Third%22&quot;&gt;Molar, Third*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Molar%2C+Third+surgery%22&quot;&gt;surgery&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Molar%2C+Third%22&quot;&gt;Molar, Third*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Molar%2C+Third+diagnostic+imaging%22&quot;&gt;diagnostic imaging&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Large+Language+Models%22&quot;&gt;Large Language Models*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Tooth+Extraction%22&quot;&gt;Tooth Extraction*&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Mandible%22&quot;&gt;Mandible&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Mandible+surgery%22&quot;&gt;surgery&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Humans%22&quot;&gt;Humans&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Retrospective+Studies%22&quot;&gt;Retrospective Studies&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Cross-Sectional+Studies%22&quot;&gt;Cross-Sectional Studies&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Observer+Variation%22&quot;&gt;Observer Variation&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Reproducibility+of+Results%22&quot;&gt;Reproducibility of Results&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Radiography%2C+Panoramic%22&quot;&gt;Radiography, Panoramic&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Female%22&quot;&gt;Female&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: The Pederson difficulty score (PDS) is widely used to assess mandibular third molar extraction difficulty, but is subject to interobserver variability because no universally accepted reference standard exists. This study investigates whether multimodal large language models (LLMs) demonstrate systematic grading tendencies comparable to those of human clinicians in the absence of a definitive standard.&lt;br /&gt;Methods: This retrospective cross-sectional study included 100 panoramic radiographs. Two LLMs (GPT and Gemini) and two oral and maxillofacial surgeons independently graded extraction difficulty using the PDS. No reference standard or gold-standard rater was designated; all raters were treated as independent assessors. Agreement was assessed using quadratic-weighted kappa and intraclass correlation coefficient values. Systematic bias was analysed using Bland-Altman plots and ordinal logistic regression. The effects of prompt language and session protocol were also evaluated.&lt;br /&gt;Results: The interexpert agreement was substantial (κ = 0.754). GPT demonstrated moderate agreement with experts (κ = 0.564-0.590), whereas Gemini exhibited fair-to-moderate agreement (κ = 0.356-0.461). Overall, the four-rater reliability was moderate (intraclass correlation coefficient = 0.553). Both LLMs displayed minimal systematic bias (GPT: -0.13; Gemini: -0.08), with no significant differences in grading tendencies compared with human experts (P &amp;gt; .05). However, sequential-session evaluation significantly reduced the scoring consistency for both models (GPT: P = .001; Gemini: P &amp;lt; .001), whereas prompt language had no significant effect on total PDS scores (GPT: P = .543; Gemini: P = .386). A component-level linguistic variation was observed only in GPT&#39;s assessment of the ramus relationship (P = .003).&lt;br /&gt;Conclusion: The examined LLMs showed grading tendencies partly comparable to those of human clinicians, without evidence of systematic directional bias; however, agreement remained below interexpert levels and was sensitive to conversational context. These findings should not be interpreted as evidence of diagnostic accuracy or clinical reliability, as no reference standard was established.&lt;br /&gt;Clinical Relevance: LLMs may provide supplementary grading information if used with independent-session protocols; however, their role in clinical decision-making requires further validation against defined outcome standards.&lt;br /&gt; (Copyright &#169; 2026 The Authors. Published by Elsevier Inc. All rights reserved.)
– Name: Abstract
  Label: Competing Interests
  Group: Ab
  Data: Conflict of interest The authors declare that they have no competing interests.
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: &lt;i&gt;Keywords: &lt;/i&gt;Artificial intelligence; Decision making; Large language models; Third molar; Tooth extraction
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: &lt;i&gt;Date Created: &lt;/i&gt;20260706 &lt;i&gt;Date Completed: &lt;/i&gt;20260916 &lt;i&gt;Latest Revision: &lt;/i&gt;20260916
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260916
– Name: PubmedCentralID
  Label: PubMed Central ID
  Group: ID
  Data: PMC13351125
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.identj.2026.109736
– Name: AN
  Label: PMID
  Group: ID
  Data: 42407142
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42407142
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.identj.2026.109736
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: 109736
    Subjects:
      – SubjectFull: Mandible surgery
        Type: general
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Retrospective Studies
        Type: general
      – SubjectFull: Cross-Sectional Studies
        Type: general
      – SubjectFull: Observer Variation
        Type: general
      – SubjectFull: Reproducibility of Results
        Type: general
      – SubjectFull: Radiography, Panoramic
        Type: general
      – SubjectFull: Female
        Type: general
      – SubjectFull: Molar, Third surgery
        Type: general
      – SubjectFull: Molar, Third diagnostic imaging
        Type: general
      – SubjectFull: Large Language Models
        Type: general
      – SubjectFull: Tooth Extraction
        Type: general
    Titles:
      – TitleFull: Assessment of Large Language Models and Expert Clinicians in Grading Mandibular Third Molar Extraction Difficulty.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yoon SM
      – PersonEntity:
          Name:
            NameFull: Yoo JH
      – PersonEntity:
          Name:
            NameFull: Kim T
      – PersonEntity:
          Name:
            NameFull: Kang JH
      – PersonEntity:
          Name:
            NameFull: Kim BC
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: 2026 Oct
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1875-595X
          Numbering:
            – Type: volume
              Value: 76
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: International dental journal
              Type: main
ResultId 1