Other/Unknown Material
Using pre-trained language models for extractive text summarisation of academic papers
| Τίτλος: | Using pre-trained language models for extractive text summarisation of academic papers |
|---|---|
| Συγγραφείς: | Hermansson, Erik, Boddien, Charlottte |
| Στοιχεία εκδότη: | Chalmers tekniska högskola / Institutionen för mekanik och maritima vetenskaper |
| Έτος έκδοσης: | 2020 |
| Συλλογή: | Chalmers Open Digital Repository (ODR - Chalmers University of Technology) / Chalmers tekniska högskola |
| Θεματικοί όροι: | natural language processing, nlp, machine learning, deep learning, automatic text summarisation, extractive summarisation, transformer, bert, roberta, xlnet |
| Περιγραφή: | Given the overwhelming amount of textual information on the internet and elsewhere, the automatic creation of high-quality summaries is becoming increasingly important. With the development of neural networks and pre-trained models within natural language processing in recent years, such as BERT and its derivatives, the field of automatic text summarisation has seen a lot of progress. These models have been pre-trained on large amounts of data for general knowledge and are then finetuned for specific tasks. Datasets are a limiting factor for training summarisation models, as they require a large amount of manual summaries to be created. Most of the current summarisation models have been trained and evaluated using textual data mostly from the news domain. However, pre-trained models, fine-tuned on data from the news domain, could potentially also be able to generalize and perform well on other data as well. The main objective of this thesis is to investigate the suitability of several pre-trained language models for automatic text summarisation. The chosen models were finetuned on readily available news data, and evaluated on a very different dataset of academic texts to determine their ability to generalise. There were only slight differences between the models on the news data. But more interestingly, the results on the academic texts showed significant differences between the models. The results indicate that the more robustly pre-trained models are able to generalise better and according to the metrics perform quite well. However, human evaluation puts this into question, showing that even the high-scoring summaries did not necessarily read well. This highlights the need for better evaluation methods and metrics. |
| Τύπος εγγράφου: | other/unknown material |
| Περιγραφή αρχείου: | application/pdf |
| Γλώσσα: | English |
| Relation: | 2020:01; https://hdl.handle.net/20.500.12380/300719 |
| Διαθεσιμότητα: | https://hdl.handle.net/20.500.12380/300719 |
| Αριθμός Καταχώρησης: | edsbas.9B2F2E65 |
| Βάση Δεδομένων: | BASE |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://hdl.handle.net/20.500.12380/300719# Name: EDS - BASE (ns324271) Category: fullText Text: View record from BASE |
|---|---|
| Header | DbId: edsbas DbLabel: BASE An: edsbas.9B2F2E65 RelevancyScore: 746 AccessLevel: 3 PubType: Other/Unknown Material PubTypeId: unknown PreciseRelevancyScore: 745.994262695313 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Using pre-trained language models for extractive text summarisation of academic papers – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Hermansson%2C+Erik%22">Hermansson, Erik</searchLink><br /><searchLink fieldCode="AR" term="%22Boddien%2C+Charlottte%22">Boddien, Charlottte</searchLink> – Name: Publisher Label: Publisher Information Group: PubInfo Data: Chalmers tekniska högskola / Institutionen för mekanik och maritima vetenskaper – Name: DatePubCY Label: Publication Year Group: Date Data: 2020 – Name: Subset Label: Collection Group: HoldingsInfo Data: Chalmers Open Digital Repository (ODR - Chalmers University of Technology) / Chalmers tekniska högskola – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22natural+language+processing%22">natural language processing</searchLink><br /><searchLink fieldCode="DE" term="%22nlp%22">nlp</searchLink><br /><searchLink fieldCode="DE" term="%22machine+learning%22">machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22deep+learning%22">deep learning</searchLink><br /><searchLink fieldCode="DE" term="%22automatic+text+summarisation%22">automatic text summarisation</searchLink><br /><searchLink fieldCode="DE" term="%22extractive+summarisation%22">extractive summarisation</searchLink><br /><searchLink fieldCode="DE" term="%22transformer%22">transformer</searchLink><br /><searchLink fieldCode="DE" term="%22bert%22">bert</searchLink><br /><searchLink fieldCode="DE" term="%22roberta%22">roberta</searchLink><br /><searchLink fieldCode="DE" term="%22xlnet%22">xlnet</searchLink> – Name: Abstract Label: Description Group: Ab Data: Given the overwhelming amount of textual information on the internet and elsewhere, the automatic creation of high-quality summaries is becoming increasingly important. With the development of neural networks and pre-trained models within natural language processing in recent years, such as BERT and its derivatives, the field of automatic text summarisation has seen a lot of progress. These models have been pre-trained on large amounts of data for general knowledge and are then finetuned for specific tasks. Datasets are a limiting factor for training summarisation models, as they require a large amount of manual summaries to be created. Most of the current summarisation models have been trained and evaluated using textual data mostly from the news domain. However, pre-trained models, fine-tuned on data from the news domain, could potentially also be able to generalize and perform well on other data as well. The main objective of this thesis is to investigate the suitability of several pre-trained language models for automatic text summarisation. The chosen models were finetuned on readily available news data, and evaluated on a very different dataset of academic texts to determine their ability to generalise. There were only slight differences between the models on the news data. But more interestingly, the results on the academic texts showed significant differences between the models. The results indicate that the more robustly pre-trained models are able to generalise better and according to the metrics perform quite well. However, human evaluation puts this into question, showing that even the high-scoring summaries did not necessarily read well. This highlights the need for better evaluation methods and metrics. – Name: TypeDocument Label: Document Type Group: TypDoc Data: other/unknown material – Name: Format Label: File Description Group: SrcInfo Data: application/pdf – Name: Language Label: Language Group: Lang Data: English – Name: NoteTitleSource Label: Relation Group: SrcInfo Data: 2020:01; https://hdl.handle.net/20.500.12380/300719 – Name: URL Label: Availability Group: URL Data: https://hdl.handle.net/20.500.12380/300719 – Name: AN Label: Accession Number Group: ID Data: edsbas.9B2F2E65 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.9B2F2E65 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English Subjects: – SubjectFull: natural language processing Type: general – SubjectFull: nlp Type: general – SubjectFull: machine learning Type: general – SubjectFull: deep learning Type: general – SubjectFull: automatic text summarisation Type: general – SubjectFull: extractive summarisation Type: general – SubjectFull: transformer Type: general – SubjectFull: bert Type: general – SubjectFull: roberta Type: general – SubjectFull: xlnet Type: general Titles: – TitleFull: Using pre-trained language models for extractive text summarisation of academic papers Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Hermansson, Erik – PersonEntity: Name: NameFull: Boddien, Charlottte IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2020 Identifiers: – Type: issn-locals Value: edsbas |
| ResultId | 1 |