Logo image
GUing: A Mobile GUI Search Engine using a Vision-Language Model
Acte de colloque

GUing: A Mobile GUI Search Engine using a Vision-Language Model

Jialiang Wei, Anne-Lise Courbis, Thomas Lambolais, Binbin Xu, Pierre Louis Bernard, Gérard Dray et Walid Maalej
Lecture Notes in Informatics Lni Proceedings Series of the Gesellschaft Fur Informatik Gi, P-377, pp. 151-152
SE 2026 - Software Engineering Conference (Bern, Switzerland, 23/02/2026–27/02/2026)
2026

Résumé

Creativity in SE AI-Inspired Design AI4SE App Development App Store Mining Computer Vision Digital Design Requirements Elicitation Vision-Language Model
We introduce GUing, a GUI search engine that allows developers to retrieve relevant app designs using natural language queries. This system is powered by GUIClip, a specialized vision-language model trained on a novel dataset of 135k screenshots and captions extracted from Google Play. By integrating both visual and textual data, this approach significantly outperforms traditional text-only models, delivering high accuracy for text-to-GUI retrieval. This work was accepted for publication at the ACM Transactions on Software Engineering and Methodology and first appeared online on November 8th 2024 [We25a].

Fichiers et liens (1)

url
Find in HALAfficher

Indicateurs

1 Consultations de la notice

Détails

Logo image