Mixed Materials
The Prevalence of Code Review Guidelines for GUI-Based Testing in Open-Source
| Title: | The Prevalence of Code Review Guidelines for GUI-Based Testing in Open-Source |
|---|---|
| Authors: | Bauer, Andreas, Angermeir, Florian, Alégroth, Emil, Anglert, Seth |
| Publisher Information: | Blekinge Tekniska Högskola, Institutionen för programvaruteknik 2025 |
| Document Type: | Mixed materials |
| Abstract: | Context: Code review has become a core practice in collaborative software engineering, helping ensure code quality, detecting potential bugs, and supporting communication among developers. Prior research has shown that code review practices differ between production and test code, suggesting that established code review guidelines may fall short in the context of test and GUI-based test code. Particularly, GUI-based testing lacks adequate support during the code review process. To address this, we proposed a set of code review guidelines specifically designed for reviewing GUI-based test files, which, however, have not yet been empirically evaluated, limiting their practical relevance. Objective: This study empirically assesses the extent to which code review comments on GUI-based tests align (explicitly or implicitly) with the concerns captured by the proposed guidelines, and uses the findings to refine the guideline set. Method: To achieve this, we sampled code review comments discussing GUI-based test files across 100 open-source projects and manually analyzed 1000 pull requests to determine to what extent the reviewers' comments align with the proposed guidelines. Results: Review comments aligned with the proposed guidelines in 808 of 1000 pull requests. We found empirical evidence for 25 of the 33 guidelines. The most frequently observed guideline concerns the correct use of testing techniques and exception handling, particularly regarding locators, explicit waits, and timeout behavior. Conclusion: The observed alignment suggests that the proposed guidelines capture concerns articulated in practice, indicating practical relevance for GUI-based test reviews. This represents an initial step towards providing empirical validation of the proposed guidelines, highlighting their potential value in enhancing the quality of GUI-based test reviews. |
| Index Terms: | Software Engineering, Programvaruteknik, Article in journal, info:eu-repo/semantics/article, text |
| DOI: | 10.2139.ssrn.5547512 |
| URL: | Information and Software Technology, 0950-5849, 2025 |
| Availability: | Open access content. Open access content info:eu-repo/semantics/openAccess |
| Note: | application/pdf English |
| Other Numbers: | UPE oai:DiVA.org:bth-28725 0000-0002-2916-4020 0000-0001-7903-8236 0000-0001-7526-3727 doi:10.2139/ssrn.5547512 1581960304 |
| Contributing Source: | UPPSALA UNIV LIBR From OAIster®, provided by the OCLC Cooperative. |
| Accession Number: | edsoai.on1581960304 |
| Database: | OAIster |
Be the first to leave a comment!