Résumé
Recently, there has been increasing interest in multimodal applications that
integrate text with other modalities, such as images, audio and video, to
facilitate natural language interactions with multimodal AI systems. While
applications involving standard modalities have been extensively explored,
there is still a lack of investigation into specific data modalities such as
remote sensing (RS) data. Despite the numerous potential applications of RS
data, including environmental protection, disaster monitoring and land
planning, available solutions are predominantly focused on specific tasks like
classification, captioning and retrieval. These solutions often overlook the
unique characteristics of RS data, such as its capability to systematically
provide information on the same geographical areas over time. This ability
enables continuous monitoring of changes in the underlying landscape. To
address this gap, we propose a novel foundation model for bi-temporal RS image
pairs, in the context of change detection analysis, leveraging Contrastive
Learning and the LEVIR-CC dataset for both captioning and text-image retrieval.
By jointly training a contrastive encoder and captioning decoder, our model add
text-image retrieval capabilities, in the context of bi-temporal change
detection, while maintaining captioning performances that are comparable to the
state of the art. We release the source code and pretrained weights at:
https://github.com/rogerferrod/RSICRC.