Academic Journal
Describing Land Cover Changes via Multi-Temporal Remote Sensing Image Captioning Using LLM, ViT, and LoRA
| Τίτλος: | Describing Land Cover Changes via Multi-Temporal Remote Sensing Image Captioning Using LLM, ViT, and LoRA |
|---|---|
| Συγγραφείς: | Lamar-Leon, Javier, Nogueira, Vitor, Salgueiro, Pedro, Quaresma, Paulo |
| Συνεισφορές: | Pan, Jiayi, Li, Xinghua |
| Στοιχεία εκδότη: | Remote Sensing MDPI |
| Έτος έκδοσης: | 2026 |
| Συλλογή: | Repositório Científico da Universidade de Évora |
| Θεματικοί όροι: | Image Captioning, Remote Sensing, LLM, LoRA |
| Περιγραφή: | Describing land cover changes from multi-temporal remote sensing imagery requires capturing both visual transformations and their semantic meaning in natural language. Existing methods often struggle to balance visual accuracy with descriptive coherence. We propose MVLT-LoRA-CC (Multi-modal Vision Language Transformer with Low-Rank Adaptation for Change Captioning), a framework that integrates a Vision Transformer (ViT), a Large Language Model (LLM), and Low-Rank Adaptation (LoRA) for efficient multi-modal learning. The model processes paired temporal images through patch embeddings and transformer blocks, aligning visual and textual representations via a multi-modal adapter. To improve efficiency and avoid unnecessary parameter growth, LoRA modules are selectively inserted only into the attention projection layers and cross-modal adapter blocks rather than being uniformly applied to all linear layers. This targeted design preserves general linguistic knowledge while enabling effective adaptation to remote sensing change description. To assess performance, we introduce the Complementary Consistency Score (CCS) framework, which evaluates both descriptive fidelity for change instances and classification accuracy for no change cases. Experiments on the LEVIR-CC test set demonstrate that MVLT-LoRA-CC generates semantically accurate captions, surpassing prior methods in both descriptive richness and temporal change recognition. The approach establishes a scalable solution for multi-modal land cover change description in remote sensing applications. |
| Τύπος εγγράφου: | article in journal/newspaper |
| Γλώσσα: | Portuguese |
| Relation: | https://hdl.handle.net/10174/41416; jlamarleon@uevora.pt; vbn@uevora.pt; pds@uevora.pt; pq@uevora.pt; 283; https://doi.org/10.3390/rs18010166 |
| DOI: | 10.3390/rs18010166 |
| Διαθεσιμότητα: | https://hdl.handle.net/10174/41416 https://doi.org/10.3390/rs18010166 |
| Rights: | openAccess |
| Αριθμός Καταχώρησης: | edsbas.EFEEF817 |
| Βάση Δεδομένων: | BASE |
| DOI: | 10.3390/rs18010166 |
|---|