Abstract
The last decade has seen the emergence and growth of deep learning techniques, leading to huge advances in computer vision, natural language processing and speech recognition.These advances are due to hardware improvements and new architectures, notably convolutional neural networks and transformers, used in popular applications such as Siri, Deepl, and ChatGPT.A major challenge in the field of computer vision was ImageNet, a database of one million images divided into 1,000 classes, used as a benchmark to measure model performance.Initial results showed a top-1 error of 37.5%, while the best models now achieve a top-1 error of 9%.However, ImageNet differs from real-life situations in that it features artificially balanced classes and few similarities between them.To address more realistic challenges, it is essential to focus on fine-grained visual categorization tasks, involving similar classes and unbalanced distributions.To this end, in this thesis we will take as a case study Pl@ntNet, an ecological application based on cooperative learning, which enables users to identify plants from images.In such a context of ambiguity, traditional multi-class classifiers, which only return a single proposition, are unable to cope.This is why, in this manuscript, we will study set-valued classifiers, which return a set of possible classes for each image.Set-valued classifiers are useful if they return a restricted number of classes for each image.Thus, there are several constraints on the size of the returned sets.In this thesis, we study two types of constraints: a pointwise constraint, where the classifier returns exactly K candidate classes for each example (top-K classification), and a constraint on the average size of the returned sets (average-K classification).To optimize these set-valued classifiers, we introduce new loss functions to improve the performance of deep learning models, one for each type of constraint.The loss function for top-K classification is based on a hinge loss function combined with a smoothing of the top-K function.For average-K classification, we propose a two-headed model, where one head is responsible for identifying candidate classes for a given example, and the other head optimizes these suggestions with binary cross-entropy.Experiments are carried out on a dataset created from Pl@ntNet data, Pl@ntNet-300K, consisting of 306,146 plant images with high class imbalance and visual ambiguities.The results show that the new loss functions significantly improve performance over cross-entropy, particularly in situations of high uncertainty.