Our paper, “AR Assistant for Pruning of Grapevines and Fruit Trees”, was presented at the Proceedings of the 21st EuroXR International Conference (EuroXR 2024).
Introduction
Winter pruning in orchards is an essential but labor-intensive and time-consuming task, traditionally done manually to shape future growth by removing unwanted branches. Workers use various techniques to optimize yield, fruit quality or disease resistance. However, labor shortages and the need for professional training pose challenges for farmers. To address this, we offer support to make pruning more accessible to a broader segment of the workforce.
At the same time, computer vision in outdoor environments is complex due to varying lighting, weather, unique plant shapes, occlusions and similarity between foreground and background plants. Multiple studies have addressed the problem of pruning grapevines and other fruit trees (Amatya, 2016; Botterill, 2017; Gentilhomme, 2023; Tong2023). These studies either focus on the entire automation pipeline (Botterill, 2017; Fourie, 2021) or separate steps, for instance, branch detection (Amatya, 2016; Zhang, 2018), reconstruction and skeletonization (You, 2022; Feng, 2024), and cut position localization (Marset, 2021). The recent approaches (Fourie, 2021; Gentilhomme, 2023) prove the general feasibility of automated pruning systems but do not address real-world challenges like outdoor conditions and complex, occluded plant structures. Effective pruning systems for fruit trees require accurate spatial information. Several studies have highlighted challenges in capturing thin structures using laser scanners or 3D cameras, often requiring additional refinement or proper initial registration, which can be time-consuming (Tagarakis 2013; Medeiros, 2017). To achieve a balance between cost and benefit, affordable methods for 3D reconstruction need to be explored. In contrast to similar studies that use 3D sensors for apple trees (Majeed, 2018; Tong, 2023), we explore the potential of image-based approaches suitable for an augmented reality (AR) pruning assistant on a mobile device.
In this work, we address the mentioned challenges and present an AR assistant that enables inexperienced workers to carry out pruning for grapevines and reduces the size of the cut wounds, making the plants more resilient to fungal infections and promoting rich and healthy yield. We further apply this concept to other fruit trees, such as apple and peach trees, and highlight the improvements made in this direction. Our contributions can be summarized as follows. First, we present a pipeline that extracts 3D and semantic information from a video of a plant and outputs pruning suggestions using both traditional and deep-learning methods (Vid2Cuts). Second, we introduce a mobile AR application to display the results to the user. Third, we extend the pipeline for other fruit trees that are more challenging compared to grapevines due to their more complex 3D structure and larger size.
Authors
Mariia Podguzova, Simon Häring, Jamiu Ojeleye, Stephan Krauß, and Didier Stricker
Read the full publication
Access all our CORTEX2 publications.