Development of machine vision system capable of locating king flowers on apple trees

Researchers at Pennsylvania State University have devised a machine vision system that can locate and identify apple king flowers in flower clusters on orchard trees. .

Apple blossoms have 4 to 6 flowers clustered on each branch, and the central flower is called the royal flower. The flower opens first in the cluster and usually produces the largest fruit. As such, it is an important target for robotic pollination systems, according to researcher Long He, an assistant professor of agriculture and bioengineering.

Apple productivity has traditionally relied on insect pollination. But evidence suggests that pollination services by both domesticated bees and wild pollinators are not keeping up with the increasing demand, he noted. of bees are dying at an alarming rate. As a result, growers need alternative methods of pollination.

The study is the latest one conducted by He’s research group at the University of Agricultural Sciences, which performs labor-intensive agricultural tasks such as picking mushrooms, pruning apple trees, and thinning fruits and vegetables. I am working on the development of a robot system for The main goal of the project, he explained, is to develop a deep-learning-based vision system that can accurately identify and locate King His Flowers in the canopy of trees.

“We believe this result provides baseline information for a robotic pollination system that leads to efficient and reproducible pollination of apples to maximize yields of high-quality fruit,” he said. “In Pennsylvania, we can still rely on bees to pollinate apple crops, but in other areas where bee mortality is more severe, growers may sooner or later need this technology.”

Xin Yang Mu, a PhD student in the Department of Agricultural Biotechnology, spearheaded the kingflower research. Mu used his Mask R-CNN. Mask R-CNN is a popular deep-his learning computer program that performs pixel-level segmentation to detect objects that are partially obscured by other objects.

To build a Mask R-CNN based detection model, he took hundreds of photos of apple blossom clusters. He then developed a Kingflower Segmentation algorithm to identify and locate kingflowers from a raw dataset of apple blossom images. The study was conducted at Penn State University’s Center for Fruit Research and Dissemination, Biglerville.

Gala and Honeycrisp Apple varieties were selected for testing. Test trees were planted in 2014 at approximately 5 ft (gara) and 6 1/2 ft (honeycrisp) spacing. These trees were trained in a tall spindle canopy architecture with an average height of about 13 feet. An image acquisition system with a camera was mounted on a utility vehicle that moved between rows of trees.

Mu noted that training a machine vision system to find the king flower is difficult. Because the king flower is the same size, color and shape as the side flowers in the cluster, the king flower is usually hidden by the surrounding flowers due to its central location.

To meet the transfer learning requirements for Mask R-CNN model training, the raw images were labeled with two predefined classes: distinct flowers and occluded flowers. To improve accuracy, Mu explains that they used a data augmentation approach to grow the training dataset four times his.

“We targeted or localized the most central flower within each flower cluster to distinguish crown and lateral flowers,” he said. “The vision system automatically placed individual flower clusters based on a two-dimensional flower density mapping approach. Within each detected flower cluster, the flower (or mask ) was determined as the target King Flower.”

In a recently published survey, smart farming technologyThe researchers reported that Mu’s algorithm yielded a high level of king flower detection accuracy. Compared to measurements made manually by the researcher identifying the kingflower by eye (called the ground truth measurement by the researcher), machine vision’s kingflower detection accuracy varied from his 98.7% to 65.6%. .