Abstract
In agriculture, thanks to the availability of mobile devices and the development of dedicated software, digital tools assist many tasks. Among digital technologies, augmented reality, the superimposition of virtual objects on views of the real world, is a powerful tool to visualize the evolution of a plot or to plan agricultural processes by interacting with digital representations of plants. This technology already helps for many works (Zheng and Campbell 2019; Janina et al. 2018; Katsaros et al. 2017). In this contribution, we propose a novel application case for agroforestry system design workshops.Agroforestry system design workshops gather different actors (farmers, extensionists, local policymakers etc.) and aim at collectively decide the tree species to plant, the spatial organisation of the system and the tree and crop management options. Currently, such workshops use whiteboards, maps or physical mock-ups representing the system with tokens, pins or toothpicks. The result of these workshops is a model that represents part of the plot. Such workshops promote discussion between the different actors, but the participants cannot observe the impact of their choices and it is impossible to see the evolution of the plot over the years. Moreover, the model isn’t a usable plantation map, an expert must adapt it to the whole plot and translate it into a usable plantation map.In order to facilitate agroforestry system design and adoption, we propose to use augmented reality both for indoor design workshops and for outdoor field visits. For the indoor design application, our objective is to allow users to easily interact with a physical mock-up in order to facilitate user involvement and visualize quickly modification of the system by providing the users with information relevant to them: tree size, level of tree-crop competition, crop yield heterogeneity, etc. For example, a user could propose a specific tree species and visualize the consequences for the crops, as well as the agronomical or environmental performance of this system. For field visits, our aim is to visualise the impact of trees on the landscape, which would be useful for farmers to help them imagine their system, and for educational purposes, to foster discussions on the impacts of trees on crops, biodiversity, farming operations etc. Finally, the link between the indoor workshop and the in situ visualization, i.e. the implementation of the theoretical pattern in a particular plot of the farm to get the coordinates of the trees, could also benefit from augmented reality, thanks to automatic detection of the agroforestry pattern, replication of this pattern within field borders (with constraints) and generation of geographic coordinates of each tree.Thus, we identified three steps, in which augmented reality could facilitate discussions, projections and help decision-making in the process of agroforestry system design:- In the, mock-up design stage, augmented reality allows visualizing the evolution of the plants and their constraints (tree growth, competition on light, spreading of diseases…),- In the mock-up implementation stage, the mock-up instantiation leads to the definition of a 3D scene, representing the future field. This implementation stage involves several processes such as pattern recognition, graph modelling, replication, and georeferencing- At the in situ visualization stage, augmented reality is mobilized for realistic plants (trees, crops) visualisation in the plot including the possibility to simulate plant growth to see the impact on the landscape after 5, 10 or 50 years.We get some results for the two first steps. Figure 1 shows, on the left, an image acquired by a smartphone or a webcam, of a (simplified) agroforestry system mock-up, with 6 "P" paper markers representing poplar trees, 2 "C" markers representing crops and one "TRI" marker defining the reference coordinate system(trihedron). Augmented reality allows to overlay, in real-time, on this image virtual representations of the trees (green points represent the canopy as the trunk cannot be seen from this viewing angle), crops (currently restricted to the marker surface but the objective is to fill the whole alley between trees) and the coordinate system vectors. On the right part of Figure1, the mock-up has been automatically translated into a 3D scene in the Unity visualisation environment for further analysis.We propose here a methodology to move from the design stage to the mock-up implementation. It must integrate a complete description of the plot scene extracted from the image (which include to deal with the approximate marker positioning,) and extract automatically relationships such as "the elements P2, P6 and P8 form a line" or "C1 is between the left line of trees and the middle line". We will then use them to infer the Ecosystem Service Functional Motif (ESFM) (Rafflegeau et al. 2019) which is the smallest pattern supporting all the ecosystem services targeted by the designers of the agroforestry system. During the instantiation step, the system repeats the EFSM automatically over the whole plot. Anexpert could correct, if necessary, to finalize the plantation map.Further work will focus on: (i) predicting useful information (such as light or water competition) from the ESFM using simulation models and visualising them as augmented reality on the mock-up; (ii) formalising constraints in the implementation step (e.g. no trees in small corners of the plot) to replicate the ESFM throughout the map of the actual plot; (iii) visualising realistic-looking trees in situ, directly in the farmer's fields.