Academic Journal
RadCLIP: Enhancing Radiologic Image Analysis Through Contrastive Language-Image Pretraining.
| Title: | RadCLIP: Enhancing Radiologic Image Analysis Through Contrastive Language-Image Pretraining. |
|---|---|
| Authors: | Lu Z, Li H, Parikh NA, Dillman JR, He L |
| Source: | IEEE transactions on neural networks and learning systems [IEEE Trans Neural Netw Learn Syst] 2025 Oct; Vol. 36 (10), pp. 17613-17622. |
| Publication Type: | Journal Article; Research Support, N.I.H., Extramural; Research Support, Non-U.S. Gov't |
| Language: | English |
| Journal Info: | Publisher: Institute of Electrical and Electronics Engineeers Country of Publication: United States NLM ID: 101616214 Publication Model: Print Cited Medium: Internet ISSN: 2162-2388 (Electronic) Linking ISSN: 2162237X NLM ISO Abbreviation: IEEE Trans Neural Netw Learn Syst Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Piscataway, NJ : Institute of Electrical and Electronics Engineeers |
| MeSH Terms: | Image Processing, Computer-Assisted*/methods , Artificial Intelligence* , Language*, Humans ; Algorithms ; Neural Networks, Computer |
| Abstract: | The integration of artificial intelligence (AI) with radiology signifies a transformative era in medicine. Vision foundation models have been adopted to enhance radiologic imaging analysis. However, the inherent complexities of 2D and 3D radiologic data present unique challenges that existing models, which are typically pretrained on general nonmedical images, do not adequately address. To bridge this gap and harness the diagnostic precision required in radiologic imaging, we introduce radiologic contrastive language-image pretraining (RadCLIP): a cross-modal vision-language foundational model that utilizes a vision-language pretraining (VLP) framework to improve radiologic image analysis. Building on the contrastive language-image pretraining (CLIP) approach, RadCLIP incorporates a slice pooling mechanism designed for volumetric image analysis and is pretrained using a large, diverse dataset of radiologic image-text pairs. This pretraining effectively aligns radiologic images with their corresponding text annotations, resulting in a robust vision backbone for radiologic imaging. Extensive experiments demonstrate RadCLIP's superior performance in both unimodal radiologic image classification and cross-modal image-text matching, underscoring its significant promise for enhancing diagnostic accuracy and efficiency in clinical settings. Our key contributions include curating a large dataset featuring diverse radiologic 2D/3D image-text pairs, pretraining RadCLIP as a vision-language foundation model on this dataset, developing a slice pooling adapter with an attention mechanism for integrating 2D images, and conducting comprehensive evaluations of RadCLIP on various radiologic downstream tasks. |
| References: | Comput Methods Programs Biomed. 2023 Oct;240:107685. (PMID: 37429247) IEEE Trans Neural Netw Learn Syst. 2024 Dec;35(12):17973-17986. (PMID: 37713222) IEEE Trans Neural Netw Learn Syst. 2025 Jan;36(1):625-637. (PMID: 37988204) Cureus. 2023 Jul 6;15(7):e41435. (PMID: 37546142) Med Image Anal. 2017 Dec;42:60-88. (PMID: 28778026) IEEE Trans Neural Netw Learn Syst. 2022 Feb;33(2):798-810. (PMID: 33090960) Radiol Artif Intell. 2019 Jan 30;1(1):e180041. (PMID: 33937785) Lancet Oncol. 2024 Jul;25(7):879-887. (PMID: 38876123) IEEE Trans Neural Netw Learn Syst. 2024 Dec;35(12):18649-18660. (PMID: 37815962) Sci Data. 2023 Jan 19;10(1):41. (PMID: 36658144) IEEE Trans Neural Netw Learn Syst. 2024 Dec;35(12):17779-17789. (PMID: 37738193) Radiol Artif Intell. 2020 Jan 29;2(1):e190007. (PMID: 32076662) Annu Rev Biomed Eng. 2017 Jun 21;19:221-248. (PMID: 28301734) Nature. 2023 Apr;616(7956):259-265. (PMID: 37045921) Comput Biol Med. 2023 May;157:106791. (PMID: 36958234) Sci Data. 2021 Apr 29;8(1):121. (PMID: 33927208) |
| Grant Information: | R01 EB029944 United States EB NIBIB NIH HHS; R01 EB030582 United States EB NIBIB NIH HHS |
| Entry Date(s): | Date Created: 20250528 Date Completed: 20251007 Latest Revision: 20260623 |
| Update Code: | 20260624 |
| PubMed Central ID: | PMC12498476 |
| DOI: | 10.1109/TNNLS.2025.3568036 |
| PMID: | 40434863 |
| Database: | MEDLINE |
| ISSN: | 2162-2388 |
|---|---|
| DOI: | 10.1109/TNNLS.2025.3568036 |