Academic Journal
View in EDS

Development and evaluation of a computer vision algorithm for quantification of children's microactivities.

Bibliographic Details
Title: Development and evaluation of a computer vision algorithm for quantification of children's microactivities.
Authors: Lupolt SN; Department of Environmental Health & Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. slupolt1@jhu.edu.; Risk Sciences and Public Policy Institute, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. slupolt1@jhu.edu., Zhang G; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Wang J; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Tang S; Department of Pediatrics, Johns Hopkins School of Medicine, Baltimore, MD, USA., Lyu Q; Department of Environmental Health & Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA., Cho J; Department of Neuroscience, Johns Hopkins Krieger School of Arts and Sciences, Baltimore, MD, USA., Huynh C; Department of Environmental Health & Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA., Liu Q; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Peng J; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Wang X; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Yin JO; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Yuan X; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Zhang Y; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA., Yuille AL; Department of Computer Science, Johns Hopkins Whiting School of Engineering, Baltimore, MD, USA.; Department of Cognitive Science, Johns Hopkins Krieger School of Arts and Sciences, Baltimore, MD, USA., Voegtline KM; Department of Pediatrics, Johns Hopkins School of Medicine, Baltimore, MD, USA.; Department of Obstetrics and Gynecology, Weill Cornell Medicine, New York, NY, USA.; Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA., Nachman KE; Department of Environmental Health & Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. knachman@jhu.edu.; Risk Sciences and Public Policy Institute, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. knachman@jhu.edu.; Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. knachman@jhu.edu.
Source: Journal of exposure science & environmental epidemiology [J Expo Sci Environ Epidemiol] 2026 Jul; Vol. 36 (4), pp. 725-733. Date of Electronic Publication: 2025 Oct 14.
Publication Type: Journal Article; Evaluation Study
Language: English
Journal Info: Publisher: Nature Pub. Group Country of Publication: United States NLM ID: 101262796 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1559-064X (Electronic) Linking ISSN: 15590631 NLM ISO Abbreviation: J Expo Sci Environ Epidemiol Subsets: MEDLINE
Imprint Name(s): Original Publication: New York, NY : Nature Pub. Group, c2006-
MeSH Terms: Environmental Exposure*/analysis , Artificial Intelligence* , Video Recording* , Infant Behavior*, Soil/chemistry ; Dust/analysis ; Humans ; Infant ; Play and Playthings ; Mouth ; Baltimore ; Evaluation Studies as Topic ; Algorithms
Abstract: Background: Estimates of microactivity (e.g., hand- and object-to-mouth contact) frequencies are essential for modeling children's environmental exposures but are challenging to obtain due to the time and human costs of manually labeling behaviors from pre-recorded videos.
Objectives: We aim to develop and evaluate a computer vision model to quantify microactivities for young children.
Methods: The vision model was trained and validated using video footage (collected via four concurrent Go-Pro cameras) of 25 children 6-18 months playing in their homes in Baltimore, MD. We leveraged computer vision techniques to develop an algorithm to assess children's pose by identifying and tracking 3D key points (e.g., locations of children's eyes, hands, wrists, elbows, etc.). We enabled automatic measurement to track the distance between the child's hands and mouth in every video frame. When the distance reached a minimum threshold, the model logged a "contact event." We compared the timing and number of events for three microactivities (left- and right-hand-to-mouth, and object-to-mouth) yielded by the vision model to the outputs from comparable human behavioral coding.
Results: Our method recognizes children's microactivities. The timing and number of contact events detected were accurate (96-99%) on a second-level basis with minimal counting errors (<0.04-2.18 per video). We observed higher rates of object-to-mouth contacts (mean = 27 contacts/h) compared to hand-to-mouth contacts (mean = 3 contacts/h).
Impact: This study developed and evaluated a computer vision method for accurately identifying and quantifying young children's hand-to-mouth and object-to-mouth contacts from collected video, greatly reducing the costs and burden of generating microactivity data needed for soil and dust exposure modeling.
(© 2025. The Author(s), under exclusive licence to Springer Nature America, Inc.)
Competing Interests: Competing interests: The authors declare no competing interests. Ethical approval: This study has been approved by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB00020023). All methods were performed in accordance with relevant guidelines and regulations.
References: Zartarian V, Xue J, Tornero-Velez R, Brown J. Children’s lead exposure: a multimedia modeling analysis to guide public health decision-making. Environ Health Perspect. 2017;125:097009. (PMID: 10.1289/EHP1605289340965915183)
US EPA. Exposure Factors Handbook Chapter 5 2017 [Available from: https://www.epa.gov/expobox/exposure-factors-handbook-chapter-5 .
Panagopoulos Abrahamsson D, Sobus JR, Ulrich EM, Isaacs K, Moschet C, Young TM, et al. A quest to identify suitable organic tracers for estimating children’s dust ingestion rates. J Expo Sci Environ Epidemiol. 2021;31:70–81. (PMID: 10.1038/s41370-020-0244-032661335)
Ferguson A, Adelabu F, Solo-Gabriele H, Obeng-Gyasi E, Fayad-Martinez C, Gidley M, et al. Methodologies for the collection of parameters to estimate dust/soil ingestion for young children. Front Public Health. 2024;12.
Xue J, Zartarian V, Moya J, Freeman N, Beamer P, Black K, et al. A meta-analysis of children’s hand-to-mouth frequency data for estimating nondietary ingestion exposure. Risk Anal. 2007;27:411–20. (PMID: 10.1111/j.1539-6924.2007.00893.x17511707)
Tsou M-C, Özkaynak H, Beamer P, Dang W, Hsi H-C, Jiang C-B, et al. Mouthing activity data for children aged 7 to 35 months in Taiwan. J Expo Sci Environ Epidemiol. 2015;25:388–98. (PMID: 10.1038/jes.2014.5025027450)
Freeman NCG, Jimenez M, Reed KJ, Gurunathan S, Edwards RD, Roy A, et al. Quantitative analysis of children’s microactivity patterns: the Minnesota Children’s Pesticide Exposure Study. J Expo Sci Environ Epidemiol. 2001;11:501–9. (PMID: 10.1038/sj.jea.7500193)
Tulve NS, Suggs JC, McCurdy T, Cohen Hubal EA, Moya J. Frequency of mouthing behavior in young children. J Expo Sci Environ Epidemiol. 2002;12:259–64. (PMID: 10.1038/sj.jea.7500225)
Rochat P (ed.) Object manipulation and exploration in 2-to 5-month-old infants 2001.
Ruff HA. Infants’ manipulative exploration of objects: Effects of age and object characteristics. Dev Psychol. 1984;20:9–20. (PMID: 10.1037/0012-1649.20.1.9)
Palmer CF. The discriminating nature of infants’ exploratory actions. Dev Psychol. 1989;25:885–93. (PMID: 10.1037/0012-1649.25.6.885)
Malachowski LG, Needham AW. Infants exploring objects: a cascades perspective. Adv Child Dev Behav. 2023;64:39–68. (PMID: 10.1016/bs.acdb.2022.11.00137080674)
Whyte VA, McDonald PV, Baillargeon R, Newell KM. Mouthing and grasping of objects by young infants. Ecol Psychol. 1994;6:205–18. (PMID: 10.1207/s15326969eco0603_3)
Moya J, Phillips L. A review of soil and dust ingestion studies for children. J Expo Sci Environ Epidemiol. 2014;24:545–54. (PMID: 10.1038/jes.2014.1724691008)
Beamer PI, Canales RA, Bradman A, Leckie JO. Farmworker children’s residential non-dietary exposure estimates from micro-level activity time series. Environ Int. 2009;35:1202–9. (PMID: 10.1016/j.envint.2009.08.003197447132775084)
Beamer P, Key ME, Ferguson AC, Canales RA, Auyeung W, Leckie JO. Quantified activity pattern data from 6 to 27-month-old farmworker children for use in exposure assessment. Environ Res. 2008;108:239–46. (PMID: 10.1016/j.envres.2008.07.007187231682613792)
Black K, Shalat SL, Freeman NCG, Jimenez M, Donnelly KC, Calvin JA. Children’s mouthing and food-handling behavior in an agricultural community on the US/Mexico border. J Expo Sci Environ Epidemiol. 2005;15:244–51. (PMID: 10.1038/sj.jea.7500398)
Tsou M-C, Özkaynak H, Beamer P, Dang W, Hsi H-C, Jiang C-B, et al. Mouthing activity data for children age 3 to <6 years old and fraction of hand area mouthed for children age <6 years old in Taiwan. J Expo Sci Environ Epidemiol. 2018;28:182–92. (PMID: 10.1038/jes.2016.8728120832)
Kwong LH, Ercumen A, Pickering AJ, Unicomb L, Davis J, Luby SP. Age-related changes to environmental exposure: variation in the frequency that young children place hands and objects in their mouths. J Expo Sci Environ Epidemiol. 2020;30:205–16. (PMID: 10.1038/s41370-019-0115-830728484)
Ferguson AC, Canales RA, Beamer P, Auyeung W, Key M, Munninghoff A, et al. Video methods in the quantification of children’s exposures. J Expo Sci Environ Epidemiol. 2006;16:287–98. (PMID: 10.1038/sj.jea.750045916249797)
Juberg DR, Alfano K, Coughlin RJ, Thompson KM. An observational study of object mouthing behavior by young children. Pediatrics. 2001;107:135–42. (PMID: 10.1542/peds.107.1.13511134447)
Zartarian VG, Ferguson AC, Ong CG, Leckie JO. Quantifying videotaped activity patterns: video translation software and training methodologies. J Expo Anal Environ Epidemiol. 1997;7:535–42. (PMID: 9306236)
Zartarian VG, Streicker J, Rivera A, Cornejo CS, Molina S, Valadez OF, et al. A pilot study to collect micro-activity data of two- to four-year-old farm labor children in Salinas Valley, California. J Expo Anal Environ Epidemiol. 1995;5:21–34. (PMID: 7663147)
Ferguson A, Dwivedi A, Adelabu F, Ehindero E, Lamssali M, Obeng-Gyasi E, et al. Quantified activity patterns for young children in beach environments relevant for exposure to contaminants. Int J Environ Res Public Health. 2021;18.
Oh HS, Ryu M. Hand-to-face contact of preschoolers during indoor activities in childcare facilities in the Republic of Korea. Int J Environ Res Public Health. 2022;19:13282. (PMID: 10.3390/ijerph192013282362938619603519)
Fang H-S, Li J, Tang H, Xu C, Zhu H, Xiu Y, et al. Alphapose: whole-body regional multi-person pose estimation and tracking in real-time. IEEE Trans Pattern Anal Mach Intell. 2022;45:7157–73. (PMID: 10.1109/TPAMI.2022.3222784)
Wang J, Sun K, Cheng T, Jiang B, Deng C, Zhao Y, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell. 2020;43:3349–64. (PMID: 10.1109/TPAMI.2020.2983686)
Loper M, Mahmood N, Romero J, Pons-Moll G, Black MJ. SMPL: a skinned multi-person linear model. Semin Graph Pap: Push Bound. 2023;2:851–66. p.
Huang X, Fu N, Liu S, Ostadabbas S (editors) Invariant representation learning for infant pose estimation with small data. In: Proceedings 16th international conference on automatic face and gesture recognition (FG 2021); IEEE; 2021.
Cai Z, Yin W, Zeng A, Wei C, Sun Q, Yanjun W, et al. Smpler-X: scaling up expressive human pose and shape estimation. Adv Neural Inf Process Syst. 2024;36.
Goel S, Pavlakos G, Rajasegaran J, Kanazawa A, Malik J. Humans in 4D: reconstructing and tracking humans with transformers. In: Proceedings IEEE/CVF international conference on computer vision (ICCV). 2023. pp 14783–94.
BuildClinical. 2025 [ https://www.buildclinical.com/ ].
Szeliski R. Computer vision: algorithms and applications, 2nd ed. Switzerland: Springer; 2022.
Joo HL, Liu H, Tan L, Gui L, Nabbe B, Matthews I, et al. Panoptic studio: a massively multiview system for social motion capture. In: Proceedings IEEE international conference on computer vision. 2015:3334–42.
Dong JFQ, Jiang W, Yang Y, Huang Q, Bao H, Zhou X. Fast and robust multi-person 3 d pose estimation and tracking from multiple views. IEEE Trans Pattern Anal Mach Intell. 2021;44:6981–92. (PMID: 10.1109/TPAMI.2021.3098052)
Ren S, He K, Girshick R, Sun J. Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;9199:2969239–50.
Contributors M. Openmmlab pose estimation toolbox and benchmark. 2020.
Sun K, Xiao B, Liu D, Wang J, editors. Deep high-resolution representation learning for human pose estimation. In: Proceedings IEEE/CVF conference on computer vision and pattern recognition; 2019.
Jin S, Xu L, Xu J, Wang C., Liu W, Qian C, et al. Whole-body human pose estimation in the wild. In: Proceedings 16th European conference on computer vision–ECCV 2020; 23–28 August; Glasgow, UK: Springer International Publishing; 2020. pp 196–214.
Ren T, Liu S, Zeng A, Lin J, Li K, Cao H, et al. Grounded Sam: assembling open-world models for diverse visual tasks. Preprint at https://doi.org/10.48550/arXiv.2401.14159 .
Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, et al., editors. Segment anything. In: Proceedings IEEE/CVF international conference on computer vision; 2023.
Easymocap—make human motion capture easier Github2021 https://github.com/zju3dv/EasyMocap .
He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016.
Bakeman R, Quera V. Behavioral observation. APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics. APA handbooks in psychology®. Washington, DC, US: American Psychological Association; 2012. p. 207–25.
Bakeman R. Behavioral observation and coding. Handbook of research methods in social and personality psychology. New York, NY, US: Cambridge University Press; 2000. p. 138–59.
Beamer PI, Luik CE, Canales RA, Leckie JO. Quantified outdoor micro-activity data for children aged 7–12-years old. J Expo Sci Environ Epidemiol. 2012;22:82–92. (PMID: 10.1038/jes.2011.3421989500)
Lopez-Galvez N, Claude J, Wong P, Bradman A, Hyland C, Castorina R, et al. Quantification and analysis of micro-level activities data from children aged 1-12 years old for use in the assessments of exposure to recycled tire on turf and playgrounds. Int J Env Res Public Health. 2022;19:2483.
Groot EM, Lekkerkerk MC, Steenbekkers, LPA. Mouthing behaviour of young children; an observational study (summary report). RIVM report 613320 002. RIVM: Bilthoven, The Netherlands; 1998.
Davis S MP, Kohler E, Wiggins C. Soil ingestion in children with PICA: Final Report (US EPA Cooperative Agreement CR 816334-01). Seattle, WA: Fred Hutchison Cancer Research Center; 1995.
Hubal EAC, Sheldon LS, Burke JM, McCurdy TR, Berry MR, Rigas ML, et al. Children’s exposure assessment: a review of factors influencing Children’s exposure, and the data available to characterize and assess that exposure. Environ Health Perspect. 2000;108:475–86. (PMID: 10.1289/ehp.00108475)
Pacheco C, Mavroudi E, Kokkoni E, Tanner HG, Vidal R, editors. A detection-based approach to multiview action classification in infants. In: Proceedings 25th international conference on pattern recognition (ICPR); 2021.
Chorney JM, McMurtry CM, Chambers CT, Bakeman R. Developing and modifying behavioral coding schemes in pediatric psychology: a practical guide. J Pediatr Psychol. 2014;40:154–64. (PMID: 10.1093/jpepsy/jsu099254168374288308)
Dechemi A, Bhakri V, Sahin I, Modi A, Mestas J, Peiris P, et al., editors. BabyNet: a lightweight network for infant reaching action recognition in unconstrained environments to support future pediatric rehabilitation applications. In: Proceedings 30th IEEE international conference on robot & human interactive communication (RO-MAN); 8–12 August 2021.
Manne SKR, Zhu S, Ostadabbas S, Wan M, editors. Automatic infant respiration estimation from video: a deep flow-based algorithm and a novel public benchmark. In: Proceedings international workshop on preterm, perinatal and paediatric image analysis. Springer; 2023.
Zhu S, Wan M, Hatamimajoumerd E, Jain K, Zlota S, Kamath CV, et al., editors. A video-based end-to-end pipeline for non-nutritive sucking action recognition and segmentation in young infants. In: Proceedings medical image computing and computer assisted intervention—MICCAI 2023; Cham: Springer Nature Switzerland; 2023.
Hesse N, Pujades S, Romero J, Black MJ, Bodensteiner C, Arens M, et al., editors. Learning an infant body model from RGB-D data for accurate full-body motion analysis. In: Proceedings 21st international conference on medical image computing and computer-assisted intervention–MICCAI 2018, Granada, Spain, 16–20 September 2018, Springer.
Xue J, Zartarian V, Tulve N, Moya J, Freeman N, Auyeung W, et al. A meta-analysis of children’s object-to-mouth frequency data for estimating non-dietary ingestion exposure. J Expo Sci Environ Epidemiol. 2010;20:536–45. (PMID: 10.1038/jes.2009.4219773815)
Grant Information: P30 ES032756 United States ES NIEHS NIH HHS
Contributed Indexing: Keywords: computer vision; dust; incidental ingestion; microactivity; soil; videography
Substance Nomenclature: 0 (Soil)
0 (Dust)
Entry Date(s): Date Created: 20251014 Date Completed: 20260730 Latest Revision: 20260730
Update Code: 20260730
PubMed Central ID: PMC13067183
DOI: 10.1038/s41370-025-00814-x
PMID: 41087742
Database: MEDLINE
Description
ISSN:1559-064X
DOI:10.1038/s41370-025-00814-x