Academic Journal
Remote data collection and cognitive task performance in the age of internet bad actors.
| Τίτλος: | Remote data collection and cognitive task performance in the age of internet bad actors. |
|---|---|
| Συγγραφείς: | O'Laughlin KD; Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States., Volponi JJ; Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States., Cheng BH; Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Menlo Education Research, LLC, Menlo Park, CA, United States., Gazzaley A; Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Departments of Neurology, Psychiatry and Physiology, University of California, San Francisco, San Francisco, CA, United States., Anguera JA; Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Department of Psychiatry, University of California, San Francisco, San Francisco, CA, United States. Electronic address: joaquin.anguera@ucsf.edu. |
| Πηγή: | Acta psychologica [Acta Psychol (Amst)] 2026 Aug; Vol. 268, pp. 107378. Date of Electronic Publication: 2026 Jul 09. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: North Holland Publishing Country of Publication: Netherlands NLM ID: 0370366 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1873-6297 (Electronic) Linking ISSN: 00016918 NLM ISO Abbreviation: Acta Psychol (Amst) Subsets: MEDLINE |
| Imprint Name(s): | Publication: Amsterdam : North Holland Publishing Original Publication: The Hague. |
| Ιατρικοί όροι (MeSH): | Cognition*/physiology , Data Collection*/methods , Executive Function*/physiology , Internet* , Task Performance and Analysis*, Memory, Short-Term/physiology ; Humans ; Female ; Male ; Adult ; Young Adult ; Neuropsychological Tests ; Digital Media |
| Περίληψη: | Bad actors and bots present threats to data integrity in remote research designs, compromising estimates of cognitive task performance. Here, we share our experiences and recommendations for identifying bad actors and bots and preventing them from enrolling in remote studies of cognitive task performance. As part of a separate large-scale validation study, 3488 participants were recruited through various platforms, including social media and MTurk, and were compensated for their participation. When inspecting these data, we observed a high incidence of unusual or nonsensical responses to demographics questions in combination with unusual cognitive performance patterns. In response, participants were categorized according to bad actor status as determined by 1) number and risk-level associated with red flags within initial screening and pattern of responses; and 2) a researcher review process to identify patterns of aberrant, patterned, or inconsistent responding. Participants were either automatically labeled as bad actors (n = 2829), labeled as bad actors after researcher review (n = 242), or labeled as good actors after researcher review (n = 417). Cognitive task performance was assessed with Adaptive Cognitive Evaluation-Explorer (ACE-X), a mobile, gamified collection of executive function tasks including measures of working memory, attention, and cognitive flexibility. We considered differences between good and bad actors for mean response times, variability in response times, accuracy, and number of trials attempted. Across 6 of the 7 ACE-X tasks evaluated, bad actors responded slower than good actors on average (β = 34.10-150.15, p < 0.001-0.047). Additional differences in task performance suggested that overall, bad actors also tended to be more variable in their response times (β = -0.32-1.01, p < 0.001), less accurate (β = -2.15-0.00, p < 0.001- > 0.999), and responded to fewer trials (β = -1.30-0.00, p < 0.001- > 0.999). Findings presented here suggest that bad actors show consistent differences in their patterns of performance on cognitive assessments, and therefore careful consideration related to study design and protocols to ensure data integrity are needed when evaluating cognitive performance in remote research settings. (Copyright © 2026 The Authors. Published by Elsevier B.V. All rights reserved.) |
| Competing Interests: | Declaration of competing interest None declared. |
| Contributed Indexing: | Keywords: Bad actors; Cognitive assessment; Data collection; Digital; Remote trial |
| Entry Date(s): | Date Created: 20260708 Date Completed: 20260724 Latest Revision: 20260724 |
| Update Code: | 20260724 |
| DOI: | 10.1016/j.actpsy.2026.107378 |
| PMID: | 42419019 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 1873-6297 |
|---|---|
| DOI: | 10.1016/j.actpsy.2026.107378 |