Academic Journal
View in EDS

Clinical note comparison and data retrieval via embedding vectors: model selection, metrics, and convergence.

Bibliographic Details
Title: Clinical note comparison and data retrieval via embedding vectors: model selection, metrics, and convergence.
Authors: Dahlberg A; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Harjun terveys, Lahti, Finland. Electronic address: alexandra.dahlberg@helsinki.fi., Tapiola O; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland., Luisto R; Faculty of Information Technology, University of Jyväskylä, Finland; Digital Workforce Services Plc, Helsinki, Finland., Puranen T; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland., Sanmark E; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland., Vartiainen V; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland.
Source: International journal of medical informatics [Int J Med Inform] 2026 Oct; Vol. 219, pp. 106563. Date of Electronic Publication: 2026 Jun 23.
Publication Type: Journal Article; Comparative Study
Language: English
Journal Info: Publisher: Elsevier Science Ireland Ltd Country of Publication: Ireland NLM ID: 9711057 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1872-8243 (Electronic) Linking ISSN: 13865056 NLM ISO Abbreviation: Int J Med Inform Subsets: MEDLINE
Imprint Name(s): Original Publication: Shannon, Co. Clare, Ireland : Elsevier Science Ireland Ltd., c1997-
MeSH Terms: Information Storage and Retrieval*/methods , Semantics* , Natural Language Processing* , Electronic Health Records*, Humans ; Generative Artificial Intelligence ; Artificial Intelligence ; Finland
Abstract: Background: Embedding models are an integral part of generative AI architectures, transforming text into embedding vectors that represent semantic content in numerical form. Despite their central role, their performance in clinical settings remains underexplored. We evaluated embedding models across two tasks: semantic difference detection in clinical notes, and data retrieval from patient records.
Methods: Eight models were applied to synthetic discharge summaries in English, Swedish, and Finnish. Semantic sensitivity was assessed by introducing controlled perturbations (deletion, modification, and paraphrasing) at three levels of severity; cosine similarity, L1 and Euclidean distances were computed between the vectors of the original and perturbed texts. Partial vectors were compared to explore dimensionality reduction. Two models with the biggest contrast in semantic difference detection were evaluated on retrieval of relevant information from real Finnish vascular surgery records.
Results: Embedding vectors captured semantic differences in clinical notes: content deletion and modification produced larger increases in vector distance than paraphrasing. On average, models detected the direction of semantic change correctly, but case-level performance varied considerably. Qwen3-Embedding-8B produced no case-level (directional) errors whereas multilingual-E5-large produced them the most (12.2%). In retrieval this contrast transferred only partially and was task-dependent: sufficiency scores favoured Qwen3-Embedding-8B for the vascular-diagnosis question (2.25 vs 1.15 out of 5) but were comparable for the antithrombotic-medication question (3.25 vs 3.17 out of 5). For some models, as few as 0.6-1.2% of dimensions sufficed to replicate full-vector accuracy; principal component analysis and coordinate-level analysis did not account for this finding.
Conclusions: Our results show that the choice of embedding model is important: performance differences between models can be large enough to determine whether clinically relevant information reaches the end user, and model weaknesses can be both task-specific and context-dependent.
(Copyright © 2026 The Author(s). Published by Elsevier B.V. All rights reserved.)
Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Alexandra Dahlberg reports financial support was provided by Research Council of Finland. Olli Tapiola reports financial support was provided by Research Council of Finland. Rami Luisto reports financial support was provided by Research Council of Finland. Tuukka Puranen reports financial support was provided by Research Council of Finland. Enni Sanmark reports financial support was provided by Research Council of Finland. Ville Vartiainen reports financial support was provided by Research Council of Finland. Alexandra Dahlberg reports financial support was provided by Business Finland. Olli Tapiola reports financial support was provided by Business Finland. Rami Luisto reports financial support was provided by Business Finland. Tuukka Puranen reports financial support was provided by Business Finland. Enni Sanmark reports financial support was provided by Business Finland. Ville Vartiainen reports financial support was provided by Business Finland. Alexandra Dahlberg reports financial support was provided by Finska Läkaresällskapet. Alexandra Dahlberg reports financial support was provided by Frans Wilhelm och Waldermar von Frenckells understödsfond. Alexandra Dahlberg reports financial support was provided by Finnish Medical Association. Enni Sanmark reports financial support was provided by Finska Läkaresällskapet. Enni Sanmark reports financial support was provided by Finnish Medical Association. Enni Sanmark reports financial support was provided by Liv och Hälsa Medical Support Association. Alexandra Dahlberg reports a relationship with Harjun terveys that includes: employment. Rami Luisto reports a relationship with Digital Workforce Plc that includes: employment. Rami Luisto reports a relationship with QuTwo Ltd that includes: employment. Rami Luisto reports a relationship with IatroData Ltd that includes: employment. Ville Vartiainen reports a relationship with Orion Pharma that includes: speaking and lecture fees. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Contributed Indexing: Keywords: Artificial Intelligence; Clinical Documentation; Dimensionality Reduction; Embedding Models; Natural Language Processing; Semantic Similarity
Entry Date(s): Date Created: 20260625 Date Completed: 20260716 Latest Revision: 20260716
Update Code: 20260717
DOI: 10.1016/j.ijmedinf.2026.106563
PMID: 42349271
Database: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://www.doi.org/10.1016/j.ijmedinf.2026.106563?
    Name: ScienceDirect (all content) (s7799221)
    Category: fullText
    Text: View record from ScienceDirect
    MouseOverText: View record from ScienceDirect
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=18728243&ISBN=&volume=219&issue=&date=20261001&spage=106563&pages=106563&title=International journal of medical informatics&atitle=Clinical%20note%20comparison%20and%20data%20retrieval%20via%20embedding%20vectors%3A%20model%20selection%2C%20metrics%2C%20and%20convergence.&aulast=Dahlberg%20A&id=DOI:10.1016/j.ijmedinf.2026.106563
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 42349271
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Clinical note comparison and data retrieval via embedding vectors: model selection, metrics, and convergence.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AU" term="%22Dahlberg+A%22">Dahlberg A</searchLink>; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Harjun terveys, Lahti, Finland. Electronic address: alexandra.dahlberg@helsinki.fi.<br /><searchLink fieldCode="AU" term="%22Tapiola+O%22">Tapiola O</searchLink>; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland.<br /><searchLink fieldCode="AU" term="%22Luisto+R%22">Luisto R</searchLink>; Faculty of Information Technology, University of Jyväskylä, Finland; Digital Workforce Services Plc, Helsinki, Finland.<br /><searchLink fieldCode="AU" term="%22Puranen+T%22">Puranen T</searchLink>; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland.<br /><searchLink fieldCode="AU" term="%22Sanmark+E%22">Sanmark E</searchLink>; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland.<br /><searchLink fieldCode="AU" term="%22Vartiainen+V%22">Vartiainen V</searchLink>; Clinicum, Faculty of Medicine, University of Helsinki, Helsinki, Finland; Heart and Lung Center, Helsinki University Hospital, Helsinki, Finland.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%229711057%22">International journal of medical informatics</searchLink> [Int J Med Inform] 2026 Oct; Vol. 219, pp. 106563. <i>Date of Electronic Publication: </i>2026 Jun 23.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article; Comparative Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: <i>Publisher: </i><searchLink fieldCode="PB" term="%22Elsevier+Science+Ireland+Ltd%22">Elsevier Science Ireland Ltd </searchLink><i>Country of Publication: </i>Ireland <i>NLM ID: </i>9711057 <i>Publication Model: </i>Print-Electronic <i>Cited Medium: </i>Internet <i>ISSN: </i>1872-8243 (Electronic) <i>Linking ISSN: </i><searchLink fieldCode="IS" term="%2213865056%22">13865056 </searchLink><i>NLM ISO Abbreviation: </i>Int J Med Inform <i>Subsets: </i>MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: <i>Original Publication</i>: Shannon, Co. Clare, Ireland : Elsevier Science Ireland Ltd., c1997-
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: <searchLink fieldCode="MM" term="%22Information+Storage+and+Retrieval%22">Information Storage and Retrieval*</searchLink>/<searchLink fieldCode="MM" term="%22Information+Storage+and+Retrieval+methods%22">methods</searchLink> <br /><searchLink fieldCode="MM" term="%22Semantics%22">Semantics*</searchLink> <br /><searchLink fieldCode="MM" term="%22Natural+Language+Processing%22">Natural Language Processing*</searchLink> <br /><searchLink fieldCode="MM" term="%22Electronic+Health+Records%22">Electronic Health Records*</searchLink><br /><searchLink fieldCode="MH" term="%22Humans%22">Humans</searchLink> ; <searchLink fieldCode="MH" term="%22Generative+Artificial+Intelligence%22">Generative Artificial Intelligence</searchLink> ; <searchLink fieldCode="MH" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink> ; <searchLink fieldCode="MH" term="%22Finland%22">Finland</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Embedding models are an integral part of generative AI architectures, transforming text into embedding vectors that represent semantic content in numerical form. Despite their central role, their performance in clinical settings remains underexplored. We evaluated embedding models across two tasks: semantic difference detection in clinical notes, and data retrieval from patient records.<br />Methods: Eight models were applied to synthetic discharge summaries in English, Swedish, and Finnish. Semantic sensitivity was assessed by introducing controlled perturbations (deletion, modification, and paraphrasing) at three levels of severity; cosine similarity, L<superscript>1</superscript> and Euclidean distances were computed between the vectors of the original and perturbed texts. Partial vectors were compared to explore dimensionality reduction. Two models with the biggest contrast in semantic difference detection were evaluated on retrieval of relevant information from real Finnish vascular surgery records.<br />Results: Embedding vectors captured semantic differences in clinical notes: content deletion and modification produced larger increases in vector distance than paraphrasing. On average, models detected the direction of semantic change correctly, but case-level performance varied considerably. Qwen3-Embedding-8B produced no case-level (directional) errors whereas multilingual-E5-large produced them the most (12.2%). In retrieval this contrast transferred only partially and was task-dependent: sufficiency scores favoured Qwen3-Embedding-8B for the vascular-diagnosis question (2.25 vs 1.15 out of 5) but were comparable for the antithrombotic-medication question (3.25 vs 3.17 out of 5). For some models, as few as 0.6-1.2% of dimensions sufficed to replicate full-vector accuracy; principal component analysis and coordinate-level analysis did not account for this finding.<br />Conclusions: Our results show that the choice of embedding model is important: performance differences between models can be large enough to determine whether clinically relevant information reaches the end user, and model weaknesses can be both task-specific and context-dependent.<br /> (Copyright © 2026 The Author(s). Published by Elsevier B.V. All rights reserved.)
– Name: Abstract
  Label: Competing Interests
  Group: Ab
  Data: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Alexandra Dahlberg reports financial support was provided by Research Council of Finland. Olli Tapiola reports financial support was provided by Research Council of Finland. Rami Luisto reports financial support was provided by Research Council of Finland. Tuukka Puranen reports financial support was provided by Research Council of Finland. Enni Sanmark reports financial support was provided by Research Council of Finland. Ville Vartiainen reports financial support was provided by Research Council of Finland. Alexandra Dahlberg reports financial support was provided by Business Finland. Olli Tapiola reports financial support was provided by Business Finland. Rami Luisto reports financial support was provided by Business Finland. Tuukka Puranen reports financial support was provided by Business Finland. Enni Sanmark reports financial support was provided by Business Finland. Ville Vartiainen reports financial support was provided by Business Finland. Alexandra Dahlberg reports financial support was provided by Finska Läkaresällskapet. Alexandra Dahlberg reports financial support was provided by Frans Wilhelm och Waldermar von Frenckells understödsfond. Alexandra Dahlberg reports financial support was provided by Finnish Medical Association. Enni Sanmark reports financial support was provided by Finska Läkaresällskapet. Enni Sanmark reports financial support was provided by Finnish Medical Association. Enni Sanmark reports financial support was provided by Liv och Hälsa Medical Support Association. Alexandra Dahlberg reports a relationship with Harjun terveys that includes: employment. Rami Luisto reports a relationship with Digital Workforce Plc that includes: employment. Rami Luisto reports a relationship with QuTwo Ltd that includes: employment. Rami Luisto reports a relationship with IatroData Ltd that includes: employment. Ville Vartiainen reports a relationship with Orion Pharma that includes: speaking and lecture fees. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: <i>Keywords: </i>Artificial Intelligence; Clinical Documentation; Dimensionality Reduction; Embedding Models; Natural Language Processing; Semantic Similarity
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: <i>Date Created: </i>20260625 <i>Date Completed: </i>20260716 <i>Latest Revision: </i>20260716
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260717
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.ijmedinf.2026.106563
– Name: AN
  Label: PMID
  Group: ID
  Data: 42349271
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42349271
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.ijmedinf.2026.106563
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: 106563
    Subjects:
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Generative Artificial Intelligence
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Finland
        Type: general
      – SubjectFull: Information Storage and Retrieval methods
        Type: general
      – SubjectFull: Semantics
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Electronic Health Records
        Type: general
    Titles:
      – TitleFull: Clinical note comparison and data retrieval via embedding vectors: model selection, metrics, and convergence.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Dahlberg A
      – PersonEntity:
          Name:
            NameFull: Tapiola O
      – PersonEntity:
          Name:
            NameFull: Luisto R
      – PersonEntity:
          Name:
            NameFull: Puranen T
      – PersonEntity:
          Name:
            NameFull: Sanmark E
      – PersonEntity:
          Name:
            NameFull: Vartiainen V
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: 2026 Oct
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1872-8243
          Numbering:
            – Type: volume
              Value: 219
          Titles:
            – TitleFull: International journal of medical informatics
              Type: main
ResultId 1