Academic Journal
Mixed Reality and Desktop Hand Hygiene Training With Deep Learning-Based Step Recognition and Real-Time Decision Support.
| Τίτλος: | Mixed Reality and Desktop Hand Hygiene Training With Deep Learning-Based Step Recognition and Real-Time Decision Support. |
|---|---|
| Συγγραφείς: | Arif SMU, Rakhimzhanova T, Myrzakhanov A, Varol HA |
| Πηγή: | IEEE transactions on visualization and computer graphics [IEEE Trans Vis Comput Graph] 2026 Jul; Vol. 32 (7), pp. 7202-7218. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: IEEE Computer Society Country of Publication: United States NLM ID: 9891704 Publication Model: Print Cited Medium: Internet ISSN: 1941-0506 (Electronic) Linking ISSN: 10772626 NLM ISO Abbreviation: IEEE Trans Vis Comput Graph Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: New York, NY : IEEE Computer Society, c1995- |
| Ιατρικοί όροι (MeSH): | Hand Hygiene*/methods , Deep Learning* , Computer Graphics* , Decision Support Techniques*, Humans ; User-Computer Interface |
| Περίληψη: | Hand hygiene (HH) is essential for preventing healthcare-associated infections, yet conventional monitoring approaches primarily capture event occurrence and provide limited insight into procedural quality, timing, and individualized feedback. To address these limitations, we present a real-time HH training and assessment framework that combines deep-learning-based WHO step recognition with a protocol-aware decision-support engine deployed on both a Desktop LED display and a Mixed Reality (MR) headset. The system supports two complementary modes: Concurrent-Feedback Coaching (CFC), which provides real-time sequence guidance and corrective prompts, and Uncued Retention Assessment (URA), which evaluates unguided execution and summarizes detected steps and errors. To support real-time deployment, we retrained and evaluated YOLOv12+MV, TimeSformer, and TSM on four heterogeneous HH datasets (PSCUH, Jurmala, METC, and Kaggle). While several YOLOv12 variants achieved strong recognition performance, the compact YOLOv12-n+MV model provided the most favorable accuracy-efficiency trade-off for deployment, achieving F1-scores of 0.99, 0.87, 0.72, and 0.58 across Kaggle, Jurmala, METC, and PSCUH, respectively, with low computational cost. This lightweight recognizer was integrated with temporal majority voting and a protocol-aware controller to support stable closed-loop interaction on Desktop and HoloLens 2. We evaluated the framework in a controlled mixed-methods study with $N=20$N=20 participants using a $2\times 2$2×2 design (Desktop vs. MR; CFC vs. URA). Desktop yielded significantly faster, more temporally stable, and less error-prone HH performance than MR, whereas CFC reduced total completion time and URA reduced weighted mistake scores, indicating a speed-accuracy trade-off. Subjective results showed higher perceived usability for Desktop than for MR and for CFC than for URA. NASA-TLX further showed a higher workload for MR than Desktop across five subscales under counterbalancing, while URA increased perceived effort relative to CFC. Overall, these findings suggest that Desktop is more suitable when efficiency, stability, and lower workload are priorities, whereas CFC and URA can be selectively used to emphasize guided acquisition or independent recall within Desktop and MR HH training workflows. |
| Entry Date(s): | Date Created: 20260521 Date Completed: 20260623 Latest Revision: 20260624 |
| Update Code: | 20260624 |
| DOI: | 10.1109/TVCG.2026.3695895 |
| PMID: | 42166256 |
| Βάση Δεδομένων: | MEDLINE |
καταχωρήστε σχόλιο πρώτοι!