Academic Journal
Using Large Language Models to Evaluate Ethical Persuasion Text: A Measurement Modeling Approach
| Τίτλος: | Using Large Language Models to Evaluate Ethical Persuasion Text: A Measurement Modeling Approach |
|---|---|
| Γλώσσα: | English |
| Συγγραφείς: | Matt Barney, Stefanie A. Wind, Vaishak Krishna |
| Πηγή: | International Journal of Assessment Tools in Education. 2026 13(1):224-247. |
| Διαθεσιμότητα: | International Journal of Assessment Tools in Education. Pamukkale University, Faculty of Education, Kinikli Campus, Denizli 20070, Turkey. e-mail: ijate.editor@gmail.com; Web site: https://dergipark.org.tr/en/pub/ijate |
| Peer Reviewed: | Y |
| Page Count: | 24 |
| Ημερομηνία έκδοσης: | 2026 |
| Τύπος εγγράφου: | Journal Articles Reports - Research |
| Descriptors: | Artificial Intelligence, Natural Language Processing, Ethics, Performance Based Assessment, Persuasive Discourse, Measurement Techniques, Psychometrics, Computer Assisted Testing |
| ISSN: | 2148-7456 |
| Περίληψη: | As AI becomes prevalent in all stages of assessment procedures, it is essential to develop procedures to ensure that its use supports ethical and psychometrically defensible measurement. In this study, we consider how measurement principles can be directly incorporated into an ethical reasoning performance assessment in which Large Language Models (LLMs) serve as raters. We demonstrate how a measurement approach can be used to obtain defensible measures of LLM-generated text related to ethics, prompts designed to elicit text-based ethical persuasion responses, and individual learners. We demonstrate how measurement quality indicators can serve as guardrails to help mitigate potential AI-related risks that can impact learners, such as hallucinations or errors. We describe a novel approach to designing, implementing, and evaluating performance assessments with AI, with the goal of enabling effective personalized learning experiences. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Αριθμός Καταχώρησης: | EJ1495732 |
| Βάση Δεδομένων: | ERIC |
| ISSN: | 2148-7456 |
|---|