Dissertation/ Thesis
AI or LLM assisted software testing : a mapping study
| Τίτλος: | AI or LLM assisted software testing : a mapping study |
|---|---|
| Συγγραφείς: | Al Amin, Shamim |
| Συνεισφορές: | Lappeenrannan-Lahden teknillinen yliopisto LUT, Lappeenranta-Lahti University of Technology LUT |
| Έτος έκδοσης: | 2025 |
| Συλλογή: | LUTPub (LUT University / LUT yliopisto) |
| Θεματικοί όροι: | AI assisted software testing, LLM based test generation, automated test analysis, security vulnerabilities in AI generated code, static analysis and CWE mapping, ODC guided quality assessment, CI/CD pipeline integration, ethical AI in testing, reproducibility and transparency, systematic literature review (SLR), fi=Kestävää teollisuutta, innovaatioita ja infrastruktuureja|en=Industry, innovation and infrastructure, fi=School of Engineering Science, Tietotekniikka|en=School of Engineering Science, Computer Science |
| Περιγραφή: | Modern software systems are increasingly distributed, frequently updated and deployed in security-critical contexts, which reduces the effectiveness of testing approaches that rely heavily on manual effort and fragile automation. AI, particularly large language models (LLMs), is being applied to automate and support software testing activities. This study presents a systematic mapping analysis based on a Systematic Literature Review (SLR) of peer-reviewed research on AI/LLM-assisted software testing, focusing on integration strategies, adoption motivations, limitations, quality impacts and ethical considerations. Searches in IEEE Xplore, ACM Digital Library, ScienceDirect, Wiley Online Library and Google Scholar retrieved 33,572 records. After multi-stage screening and quality assessment, 34 primary studies were included and synthesized using research question driven thematic analysis. The results highlight that the current integration is mostly focused on generation. The most modern applications are those that help with unit test generation and enhancement, test suite augmentation and debugging. Other uses include finding bugs, helping with test-oracle and automating CI/CD but coverage of non-functional and specialized testing is still limited. Adoption is motivated by the need to improve coverage, reduce manual effort and accelerate feedback cycles. Key challenges include hallucinations and semantic errors in generated artifacts, weak or inconsistent oracle strategies, non-determinism and reproducibility issues and security and privacy risks associated with prompts, logs and generated code. Overall, the evidence suggests that AI/LLM assisted testing can improve software quality when supported by validation mechanisms, secure integration practices and appropriate human oversight. |
| Τύπος εγγράφου: | master thesis |
| Περιγραφή αρχείου: | fulltext |
| Γλώσσα: | English |
| Relation: | https://lutpub.lut.fi/handle/10024/171117 |
| Διαθεσιμότητα: | https://lutpub.lut.fi/handle/10024/171117 |
| Rights: | fi=Kaikki oikeudet pidätetään.|en=All rights reserved.| |
| Αριθμός Καταχώρησης: | edsbas.B6F37E3A |
| Βάση Δεδομένων: | BASE |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://lutpub.lut.fi/handle/10024/171117# Name: EDS - BASE (ns324271) Category: fullText Text: View record from BASE |
|---|---|
| Header | DbId: edsbas DbLabel: BASE An: edsbas.B6F37E3A RelevancyScore: 822 AccessLevel: 3 PubType: Dissertation/ Thesis PubTypeId: dissertation PreciseRelevancyScore: 821.935607910156 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: AI or LLM assisted software testing : a mapping study – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Al+Amin%2C+Shamim%22">Al Amin, Shamim</searchLink> – Name: Author Label: Contributors Group: Au Data: Lappeenrannan-Lahden teknillinen yliopisto LUT<br />Lappeenranta-Lahti University of Technology LUT – Name: DatePubCY Label: Publication Year Group: Date Data: 2025 – Name: Subset Label: Collection Group: HoldingsInfo Data: LUTPub (LUT University / LUT yliopisto) – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22AI+assisted+software+testing%22">AI assisted software testing</searchLink><br /><searchLink fieldCode="DE" term="%22LLM+based+test+generation%22">LLM based test generation</searchLink><br /><searchLink fieldCode="DE" term="%22automated+test+analysis%22">automated test analysis</searchLink><br /><searchLink fieldCode="DE" term="%22security+vulnerabilities+in+AI+generated+code%22">security vulnerabilities in AI generated code</searchLink><br /><searchLink fieldCode="DE" term="%22static+analysis+and+CWE+mapping%22">static analysis and CWE mapping</searchLink><br /><searchLink fieldCode="DE" term="%22ODC+guided+quality+assessment%22">ODC guided quality assessment</searchLink><br /><searchLink fieldCode="DE" term="%22CI%2FCD+pipeline+integration%22">CI/CD pipeline integration</searchLink><br /><searchLink fieldCode="DE" term="%22ethical+AI+in+testing%22">ethical AI in testing</searchLink><br /><searchLink fieldCode="DE" term="%22reproducibility+and+transparency%22">reproducibility and transparency</searchLink><br /><searchLink fieldCode="DE" term="%22systematic+literature+review+%28SLR%29%22">systematic literature review (SLR)</searchLink><br /><searchLink fieldCode="DE" term="%22fi=Kestävää+teollisuutta%22">fi=Kestävää teollisuutta</searchLink><br /><searchLink fieldCode="DE" term="%22innovaatioita+ja+infrastruktuureja|en=Industry%22">innovaatioita ja infrastruktuureja|en=Industry</searchLink><br /><searchLink fieldCode="DE" term="%22innovation+and+infrastructure%22">innovation and infrastructure</searchLink><br /><searchLink fieldCode="DE" term="%22fi=School+of+Engineering+Science%22">fi=School of Engineering Science</searchLink><br /><searchLink fieldCode="DE" term="%22Tietotekniikka|en=School+of+Engineering+Science%22">Tietotekniikka|en=School of Engineering Science</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science%22">Computer Science</searchLink> – Name: Abstract Label: Description Group: Ab Data: Modern software systems are increasingly distributed, frequently updated and deployed in security-critical contexts, which reduces the effectiveness of testing approaches that rely heavily on manual effort and fragile automation. AI, particularly large language models (LLMs), is being applied to automate and support software testing activities. This study presents a systematic mapping analysis based on a Systematic Literature Review (SLR) of peer-reviewed research on AI/LLM-assisted software testing, focusing on integration strategies, adoption motivations, limitations, quality impacts and ethical considerations. Searches in IEEE Xplore, ACM Digital Library, ScienceDirect, Wiley Online Library and Google Scholar retrieved 33,572 records. After multi-stage screening and quality assessment, 34 primary studies were included and synthesized using research question driven thematic analysis. The results highlight that the current integration is mostly focused on generation. The most modern applications are those that help with unit test generation and enhancement, test suite augmentation and debugging. Other uses include finding bugs, helping with test-oracle and automating CI/CD but coverage of non-functional and specialized testing is still limited. Adoption is motivated by the need to improve coverage, reduce manual effort and accelerate feedback cycles. Key challenges include hallucinations and semantic errors in generated artifacts, weak or inconsistent oracle strategies, non-determinism and reproducibility issues and security and privacy risks associated with prompts, logs and generated code. Overall, the evidence suggests that AI/LLM assisted testing can improve software quality when supported by validation mechanisms, secure integration practices and appropriate human oversight. – Name: TypeDocument Label: Document Type Group: TypDoc Data: master thesis – Name: Format Label: File Description Group: SrcInfo Data: fulltext – Name: Language Label: Language Group: Lang Data: English – Name: NoteTitleSource Label: Relation Group: SrcInfo Data: https://lutpub.lut.fi/handle/10024/171117 – Name: URL Label: Availability Group: URL Data: https://lutpub.lut.fi/handle/10024/171117 – Name: Copyright Label: Rights Group: Cpyrght Data: fi=Kaikki oikeudet pidätetään.|en=All rights reserved.| – Name: AN Label: Accession Number Group: ID Data: edsbas.B6F37E3A |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.B6F37E3A |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English Subjects: – SubjectFull: AI assisted software testing Type: general – SubjectFull: LLM based test generation Type: general – SubjectFull: automated test analysis Type: general – SubjectFull: security vulnerabilities in AI generated code Type: general – SubjectFull: static analysis and CWE mapping Type: general – SubjectFull: ODC guided quality assessment Type: general – SubjectFull: CI/CD pipeline integration Type: general – SubjectFull: ethical AI in testing Type: general – SubjectFull: reproducibility and transparency Type: general – SubjectFull: systematic literature review (SLR) Type: general – SubjectFull: fi=Kestävää teollisuutta Type: general – SubjectFull: innovaatioita ja infrastruktuureja|en=Industry Type: general – SubjectFull: innovation and infrastructure Type: general – SubjectFull: fi=School of Engineering Science Type: general – SubjectFull: Tietotekniikka|en=School of Engineering Science Type: general – SubjectFull: Computer Science Type: general Titles: – TitleFull: AI or LLM assisted software testing : a mapping study Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Al Amin, Shamim – PersonEntity: Name: NameFull: Lappeenrannan-Lahden teknillinen yliopisto LUT – PersonEntity: Name: NameFull: Lappeenranta-Lahti University of Technology LUT IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2025 Identifiers: – Type: issn-locals Value: edsbas |
| ResultId | 1 |