Academic Journal
ST-ConMa: a multimodal foundation framework for spatial transcriptomics via image-gene contrastive and matching learning.
| Τίτλος: | ST-ConMa: a multimodal foundation framework for spatial transcriptomics via image-gene contrastive and matching learning. |
|---|---|
| Συγγραφείς: | Yeom T; Department of AI, College of Artificial Intelligence, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Republic of Korea., Choi D; Department of AI, College of Artificial Intelligence, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Republic of Korea., Lee H; Department of AI, College of Artificial Intelligence, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Republic of Korea. |
| Πηγή: | Briefings in bioinformatics [Brief Bioinform] 2026 Sep 01; Vol. 27 (5). |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: Oxford University Press Country of Publication: England NLM ID: 100912837 Publication Model: Print Cited Medium: Internet ISSN: 1477-4054 (Electronic) Linking ISSN: 14675463 NLM ISO Abbreviation: Brief Bioinform Subsets: MEDLINE |
| Imprint Name(s): | Publication: Oxford : Oxford University Press Original Publication: London ; Birmingham, AL : H. Stewart Publications, [2000- |
| Ιατρικοί όροι (MeSH): | Gene Expression Profiling*/methods , Image Processing, Computer-Assisted*/methods , Spatial Transcriptomics* , Deep Learning* , Transcriptome*, Humans ; Algorithms |
| Περίληψη: | Spatial transcriptomics (ST) enables the spatial mapping of gene expression by combining tissue images with spot-level gene expression profiles, allowing for the analysis of gene distribution and interactions between adjacent regions. Although numerous deep learning models have been developed for ST, most existing approaches are task-specific, and multimodal foundation models with broad knowledge of ST remain scarce. While a few ST multimodal models currently exist, they have been either trained on low-resolution tissue images or developed for a specific platform, thereby limiting their generalizability when applied to diverse ST data. To address this gap, we propose ST-ConMa, a multimodal foundation model pretrained on large-scale spot-level tissue images paired with corresponding gene expression profiles using contrastive and matching learning. While conventional task-specific and foundation models generate modality-specific embeddings for images and genes separately, our model jointly learns image-gene fusion representations using a multimodal encoder equipped with cross-attention and a matching objective. Through extensive evaluations on histopathology image classification benchmarks and domain-specific tasks including gene expression prediction and spatial clustering, we demonstrate that ST-ConMa learns high-quality multimodal representations and consistently outperforms existing methods. These results indicate that ST-ConMa serves as a generalizable backbone for diverse ST approaches. (© The Author(s) 2026. Published by Oxford University Press.) |
| References: | Med Image Anal. 2024 Oct;97:103289. (PMID: 39106763) Brief Bioinform. 2024 May 23;25(4):. (PMID: 38801701) Cancer Sci. 2014 Jul;105(7):779-87. (PMID: 24754246) Aging (Albany NY). 2024 May 9;16(9):8279-8305. (PMID: 38728370) Cancer Immunol Immunother. 2016 Feb;65(2):213-22. (PMID: 26759008) J Hepatol. 2024 May;80(5):730-743. (PMID: 38199298) Nat Commun. 2025 Jan 27;16(1):1067. (PMID: 39870633) IEEE Trans Med Imaging. 2019 Feb;38(2):550-560. (PMID: 30716025) Science. 2016 Jul 1;353(6294):78-82. (PMID: 27365449) Brief Bioinform. 2023 Nov 22;25(1):. (PMID: 38145948) Genome Med. 2024 Jan 12;16(1):12. (PMID: 38217035) Science. 2019 Mar 29;363(6434):1463-1467. (PMID: 30923225) Cell. 2015 May 21;161(5):1202-1214. (PMID: 26000488) Curr Protoc Bioinformatics. 2016 Jun 20;54:1.30.1-1.30.33. (PMID: 27322403) BMC Genomics. 2022 Jun 10;23(1):434. (PMID: 35689177) Bioinformatics. 2023 Jan 1;39(1):. (PMID: 36610709) Cell. 2020 Jul 23;182(2):497-514.e22. (PMID: 32579974) Brief Bioinform. 2022 Sep 20;23(5):. (PMID: 35849101) Nat Methods. 2021 Nov;18(11):1342-1351. (PMID: 34711970) Nucleic Acids Res. 2025 Jan 6;53(D1):D886-D900. (PMID: 39607691) Nat Methods. 2022 May;19(5):567-575. (PMID: 35577957) Nat Commun. 2022 Apr 1;13(1):1739. (PMID: 35365632) Adv Neural Inf Process Syst. 2024 Dec;37:35796-35823. (PMID: 41551654) Nucleic Acids Res. 2022 Dec 9;50(22):e131. (PMID: 36250636) Genome Biol. 2016 May 06;17(1):91. (PMID: 27154141) Brief Bioinform. 2024 Sep 23;25(6):. (PMID: 39471412) Nat Methods. 2025 Jul;22(7):1568-1582. (PMID: 40442373) Genome Biol. 2024 Aug 5;25(1):206. (PMID: 39103939) Oncogene. 2000 Dec 11;19(53):6102-14. (PMID: 11156523) Science. 2016 Apr 8;352(6282):189-96. (PMID: 27124452) Front Genet. 2024 Mar 08;15:1304853. (PMID: 38525245) Nat Commun. 2023 Mar 1;14(1):1155. (PMID: 36859400) Genome Med. 2018 Jul 24;10(1):57. (PMID: 30041684) Nat Commun. 2017 Jan 16;8:14049. (PMID: 28091601) Nat Methods. 2023 Sep;20(9):1379-1387. (PMID: 37592182) Nat Commun. 2021 Oct 14;12(1):6012. (PMID: 34650042) |
| Grant Information: | RS-2025-00519063 National Research Foundation of Korea; IITP-2026)-RS-2026-25615334 Institute of Information & Communications Technology Planning & Evaluation; 2019-0-01842 AI Star Fellowship Support Program; Artificial Intelligence Graduate School Program; Gwangju Institute of Science and Technology; Government of the Republic of Korea (Ministry of Science and ICT) |
| Contributed Indexing: | Keywords: foundation model; gene expression prediction; multimodal; spatial clustering; spatial transcriptomics |
| Entry Date(s): | Date Created: 20260928 Date Completed: 20260928 Latest Revision: 20260930 |
| Update Code: | 20260930 |
| PubMed Central ID: | PMC13617578 |
| DOI: | 10.1093/bib/bbag535 |
| PMID: | 42803631 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 1477-4054 |
|---|---|
| DOI: | 10.1093/bib/bbag535 |