Academic Journal
Προβολή σε EDS

Pre-trained language models with domain knowledge for biomedical extractive summarization

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Pre-trained language models with domain knowledge for biomedical extractive summarization
Συγγραφείς: Xie, Qianqian, Bishop, Jennifer, Tiwari, Prayag, Ananiadou, Sophia
Πηγή: Xie, Q, Bishop, J, Tiwari, P & Ananiadou, S 2022, 'Pre-trained language models with domain knowledge for biomedical extractive summarization', Knowledge-Based Systems. https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub
Έτος έκδοσης: 2022
Συλλογή: The University of Manchester: Research Explorer - Publications
Θεματικοί όροι: Extractive Summarisation, Biomedical text mining, PICO elements, Pre-trained language models
Περιγραφή: Biomedical text summarization is a critical task for comprehension of an ever-growing amount of biomedical literature. Pre-trained language models (PLMs) with transformer-based architectures have been shown to greatly improve performance in biomedical text mining tasks. However, existing methods for text summarization generally fine-tune PLMs on the target corpora directly and do not consider how fine-grained domain knowledge, such as PICO elements used in evidence-based medicine, can help to identify the context needed for generating coherent summaries. To fill the gap, we propose KeBioSum, a novel knowledge infusion training framework, and experiment using a number of PLMs as bases, for the task of extractive summarization on biomedical literature. We investigate generative and discriminative training techniques to fuse domain knowledge (i.e., PICO elements) into knowledge adapters and apply adapter fusion to efficiently inject the knowledge adapters into the basic PLMs for fine-tuning the extractive summarization task. Experimental results from the extractive summarization task on three biomedical literature datasets show that existing PLMs (BERT, RoBERTa, BioBERT, and PubMedBERT) are improved by incorporating the KeBioSum knowledge adapters, and our model outperforms the strong baselines.
Τύπος εγγράφου: article in journal/newspaper
Γλώσσα: English
ISSN: 0950-7051
Relation: info:eu-repo/semantics/altIdentifier/pissn/0950-7051
Διαθεσιμότητα: https://research.manchester.ac.uk/en/publications/72f6659e-3fa1-4bb3-b570-127b03b9df8b
https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub
Rights: info:eu-repo/semantics/openAccess ; http://creativecommons.org/licenses/by/4.0/
Αριθμός Καταχώρησης: edsbas.2DE16A65
Βάση Δεδομένων: BASE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://research.manchester.ac.uk/en/publications/72f6659e-3fa1-4bb3-b570-127b03b9df8b#
    Name: EDS - BASE (ns324271)
    Category: fullText
    Text: View record from BASE
  – Url: https://www.sciencedirect.com/science/link?ref_val_fmt=info:ofi/fmt:kev:mtx:journal&svc_val_fmt=info:ofi/fmt:kev:mtx:sch_srv&rfr_dat=saltver:1&rfr_dat=origin:EBSCOLINK&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&svc.fulltext=yes&rft.issn=09507051&rft.volume=&rft.title=Pre-trained language models with domain knowledge for biomedical extractive summarization
    Name: ScienceDirect (all content) (s7799221)
    Category: fullText
    Text: View record from ScienceDirect
    MouseOverText: View record from ScienceDirect
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:edsbas&genre=article&issn=09507051&ISBN=&volume=&issue=&date=20220101&spage=&pages=&title=Xie, Q, Bishop, J, Tiwari, P & Ananiadou, S 2022, 'Pre-trained language models with domain knowledge for biomedical extractive summarization', Knowledge-Based Systems. https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub&atitle=Pre-trained%20language%20models%20with%20domain%20knowledge%20for%20biomedical%20extractive%20summarization&aulast=Xie%2C%20Qianqian&id=DOI:
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: edsbas
DbLabel: BASE
An: edsbas.2DE16A65
RelevancyScore: 920
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 920.2763671875
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Pre-trained language models with domain knowledge for biomedical extractive summarization
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Xie%2C+Qianqian%22">Xie, Qianqian</searchLink><br /><searchLink fieldCode="AR" term="%22Bishop%2C+Jennifer%22">Bishop, Jennifer</searchLink><br /><searchLink fieldCode="AR" term="%22Tiwari%2C+Prayag%22">Tiwari, Prayag</searchLink><br /><searchLink fieldCode="AR" term="%22Ananiadou%2C+Sophia%22">Ananiadou, Sophia</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: Xie, Q, Bishop, J, Tiwari, P & Ananiadou, S 2022, 'Pre-trained language models with domain knowledge for biomedical extractive summarization', Knowledge-Based Systems. https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub
– Name: DatePubCY
  Label: Publication Year
  Group: Date
  Data: 2022
– Name: Subset
  Label: Collection
  Group: HoldingsInfo
  Data: The University of Manchester: Research Explorer - Publications
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Extractive+Summarisation%22">Extractive Summarisation</searchLink><br /><searchLink fieldCode="DE" term="%22Biomedical+text+mining%22">Biomedical text mining</searchLink><br /><searchLink fieldCode="DE" term="%22PICO+elements%22">PICO elements</searchLink><br /><searchLink fieldCode="DE" term="%22Pre-trained+language+models%22">Pre-trained language models</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: Biomedical text summarization is a critical task for comprehension of an ever-growing amount of biomedical literature. Pre-trained language models (PLMs) with transformer-based architectures have been shown to greatly improve performance in biomedical text mining tasks. However, existing methods for text summarization generally fine-tune PLMs on the target corpora directly and do not consider how fine-grained domain knowledge, such as PICO elements used in evidence-based medicine, can help to identify the context needed for generating coherent summaries. To fill the gap, we propose KeBioSum, a novel knowledge infusion training framework, and experiment using a number of PLMs as bases, for the task of extractive summarization on biomedical literature. We investigate generative and discriminative training techniques to fuse domain knowledge (i.e., PICO elements) into knowledge adapters and apply adapter fusion to efficiently inject the knowledge adapters into the basic PLMs for fine-tuning the extractive summarization task. Experimental results from the extractive summarization task on three biomedical literature datasets show that existing PLMs (BERT, RoBERTa, BioBERT, and PubMedBERT) are improved by incorporating the KeBioSum knowledge adapters, and our model outperforms the strong baselines.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: article in journal/newspaper
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0950-7051
– Name: NoteTitleSource
  Label: Relation
  Group: SrcInfo
  Data: info:eu-repo/semantics/altIdentifier/pissn/0950-7051
– Name: URL
  Label: Availability
  Group: URL
  Data: https://research.manchester.ac.uk/en/publications/72f6659e-3fa1-4bb3-b570-127b03b9df8b<br />https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub
– Name: Copyright
  Label: Rights
  Group: Cpyrght
  Data: info:eu-repo/semantics/openAccess ; http://creativecommons.org/licenses/by/4.0/
– Name: AN
  Label: Accession Number
  Group: ID
  Data: edsbas.2DE16A65
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.2DE16A65
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    Subjects:
      – SubjectFull: Extractive Summarisation
        Type: general
      – SubjectFull: Biomedical text mining
        Type: general
      – SubjectFull: PICO elements
        Type: general
      – SubjectFull: Pre-trained language models
        Type: general
    Titles:
      – TitleFull: Pre-trained language models with domain knowledge for biomedical extractive summarization
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Xie, Qianqian
      – PersonEntity:
          Name:
            NameFull: Bishop, Jennifer
      – PersonEntity:
          Name:
            NameFull: Tiwari, Prayag
      – PersonEntity:
          Name:
            NameFull: Ananiadou, Sophia
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 09507051
            – Type: issn-locals
              Value: edsbas
            – Type: issn-locals
              Value: edsbas.oa
          Titles:
            – TitleFull: Xie, Q, Bishop, J, Tiwari, P & Ananiadou, S 2022, 'Pre-trained language models with domain knowledge for biomedical extractive summarization', Knowledge-Based Systems. https://doi.org/https://www.sciencedirect.com/science/article/pii/S0950705122007328?via%3Dihub
              Type: main
ResultId 1