基于深度学习的蔬菜田精准除草作业区域检测方法

李卫丽; 金小俊; 于佳琳; 陈勇

基于深度学习的蔬菜田精准除草作业区域检测方法

李卫丽^{1, 2,},
金小俊^{2, 3},
于佳琳³,
陈勇^2, ,

1.
南京航空航天大学金城学院机电工程与自动化学院，江苏　南京　211156
2.
南京林业大学机械电子工程学院，江苏　南京　210037
3.
北京大学现代农业研究院，山东　潍坊　261325

基金项目: 国家自然科学基金项目（32072498）；江苏省研究生科研与实践创新计划项目（KYCX22_1051）

详细信息

作者简介:
李卫丽（1985 — ），女，硕士，讲师，主要从事智慧农业、智能机器人技术研究，E-mail：lwl2009seu@126.com

通讯作者:
陈勇（1965 — ），男，教授，博士生导师，主要从事机电一体化研究，E-mail：chenyongjsnj@163.com

中图分类号: TP391.41
计量
- 文章访问数: 74
- HTML全文浏览量: 33
- PDF下载量: 3
- 被引次数: 0
出版历程
- 收稿日期: 2023-07-10
- 修回日期: 2023-10-05
- 网络出版日期: 2024-03-28

A Deep Learning-based Method for Detection of Precision Weeding Regions in Vegetables

LI Weili^{1, 2
,},
JIN Xiaojun^{2, 3},
YU Jialin³,
CHEN Yong^{2
, ,}

1.
Jincheng College of Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu　211156, China
2.
College of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing, Jiangsu　210037, China
3.
Peking University Institute of Advanced Agriculture Sciences, Weifang, Shandong　261325, China

摘要

摘要: 目的蔬菜生长随机，杂草种类众多。传统杂草识别算法复杂，且仅识别出杂草，未能精准确定除草作业区域。本研究以蔬菜及其伴生杂草为研究对象，拟探索一种基于深度学习的杂草识别与精准除草作业区域检测方法。方法通过将原图切分网格图像，利用深度学习模型识别蔬菜、杂草及土壤，将包含杂草的网格图像标记为除草作业区域。选取ShuffleNet、DenseNet和ResNet模型开展识别试验，并采用精度、召回率、F₁值和总体准确率、平均准确率分别对验证集和测试集进行评价分析。结果所选的3种网络模型均能较好地识别杂草和蔬菜，其中ShuffleNet为杂草识别最优模型，其对杂草的识别具有较为均衡的精确度和召回率，分别为95.5%、97%，且其识别速度也达最优，为68.37 fps，能够应用于实时杂草识别。结论本研究提出的除草作业区域检测方法具有高度的可行性和极佳的识别效果，可用于蔬菜田间杂草的精准防除。
- 蔬菜 /
- 杂草 /
- 图像处理 /
- 深度学习 /
- 作业区域检测
Abstract: Objective Vegetables grow randomly and often infested with numerous weed species. Traditional algorithms are complex in weed detection, and typically fail to localize the weeding regions precisely. In the study, a deep learning-based method was proposed to identify weeds growing in vegetables in combination with weeding regions detection. Method Firstly, the original images were cropped into grid cells (sub-images). Afterwards, the deep learning models were utilized to classify the grid cells into vegetables, weeds and soil. The grid cells containing weeds were marked as weeding regions. ShuffleNet, DenseNet, and ResNet were employed to detect and classify the grid cells. Deep learning models were evaluated in terms of precision, recall value, F₁ score, and overall and average accuracy. Result The experiment results show that the evaluated deep learning models can identify weeds and vegetables effectively. ShuffleNet model was proven to be the bestmodel, which achieves optimal balance between the precision and recall value, with the values of 95.5% and 97% respectively. Additionly, ShuffleNet achieved the highest detection speed, with the value of 68.37fps, making it suitable for real-time weed detection. Conclusion The results demonstrate that the proposed method is feasible and well-performed, which can be used for precision weeds control in vegetables.
- Vegetables /
- weeds /
- image processing /
- deep learning /
- weeding region detection

HTML全文

图 1 原图切分网格图像

Figure 1. Images sliced by grid

下载: 全尺寸图片幻灯片

图 2 精准除草作业区域检测流程图

Figure 2. The flow chat of precise weeding area detection

下载: 全尺寸图片幻灯片

图 3 不同深度学习模型的混淆矩阵图

Figure 3. The confusion matrices of different deep learning models

下载: 全尺寸图片幻灯片

图 4 精准作业区域检测结果

Figure 4. The result of precise weeding area

下载: 全尺寸图片幻灯片

图 5 被识别错误的图片

Figure 5. Images that has been wrongly classified

下载: 全尺寸图片幻灯片

表 1 深度学习数据集

Table 1. The dataset of deep learning models

样本类别 Sample category	正样本 Positive sample	负样本 Negative sample
样本类别 Sample category	青菜 Vegetable	杂草 Weed	土壤 Soil
训练集 Training dataset	3000	3000	3000
验证集 Validation dataset	500	500	500
测试集 Testing dataset	500	500	500

下载: 导出CSV

表 2 不同模型的默认超参数

Table 2. Hyper-parameters for training convolutional neural networks

模型 Neural network	批尺寸 Batch size	初始学习率 Initial learning rate	学习率调整策略 Learning rate policy	优化器 Optimizer	训练周期 Training epochs
ShuffleNet	16	0.001	LambdaLR	SGD	24
DenseNet	16	0.001	LambdaLR	SGD	24
ResNet	16	0.0001	StepLR	Adam	24

下载: 导出CSV

表 3 不同深度学习模型验证集识别结果

Table 3. Evaluation metrics of deep learning models in validation dataset

模型 Neural Network	类别 Category	精度 Precision	召回率 Recall	F₁值 F₁ score
ShuffleNet	土壤 Soil	0.978	0.946	0.967
	青菜 Vegetable	0.99	0.988	0.989
	杂草 Weed	0.955	0.97	0.962
DenseNet	土壤 Soil	0.972	0.974	0.973
	青菜 Vegetable	0.975	0.994	0.984
	杂草 Weed	0.969	0.948	0.958
ResNet	土壤 Soil	0.981	0.946	0.963
	青菜 Vegetable	0.969	0.992	0.98
	杂草 Weed	0.945	0.956	0.95

下载: 导出CSV

表 4 不同深度学习模型测试集评价数据

Table 4. Evaluation metrics of different deep learning models in testing dataset

模型 Neural network	类别 Category	总体准确率 Overall accuracy	平均准确率 Average accuracy	网格图像识别速度fps Fps of grid cells	原图识别速度 48/fps Fps of full images
ShuffleNet	土壤 Soil	0.967	0.951	207.45	68.37
	青菜 Vegetable	0.978
	杂草 Weed	0.957
DenseNet	土壤 Soil	0.967	0.949	104.05	58.94
	青菜 Vegetable	0.979
	杂草 Weed	0.953
ResNet	土壤 Soil	0.962	0.941	289.57	85.42
	青菜 Vegetable	0.977
	杂草 Weed	0.944

下载: 导出CSV

参考文献(36)

[1]	金月, 肖宏儒, 曹光乔, 等. 我国叶类蔬菜机械化水平现状与评价方法研究 [J]. 中国农机化学报, 2020, 41(12):196−201. JIN Y, XIAO H R, CAO G Q, et al. Research on status and evaluation methods of leafy vegetable mechanization production level in China [J]. Journal of Chinese Agricultural Mechanization, 2020, 41(12): 196−201. (in Chinese)
[2]	中华人民共和国国家统计局. 农业年度数据 [EB/OL]. http://data.stats.gov.cn/easyquery.htm?cn=C01.
[3]	刘文, 徐丽明, 邢洁洁, 等. 作物株间机械除草技术的研究现状 [J]. 农机化研究, 2017, 39(1):243−250. LIU W, XU L M, XING J J, et al. Research status of mechanical intra-row weed control in row crops [J]. Journal of Agricultural Mechanization Research, 2017, 39(1): 243−250. (in Chinese)
[4]	强胜. 我国杂草学研究现状及其发展策略 [J]. 植物保护, 2010, 36(4):1−5. doi: 10.3969/j.issn.0529-1542.2010.04.001 QIANG S. Current status and development strategy for weed science in China [J]. Plant Protection, 2010, 36(4): 1−5. (in Chinese) doi: 10.3969/j.issn.0529-1542.2010.04.001
[5]	陈德润, 王书茂, 王秀. 农田杂草识别技术的现状与展望 [J]. 中国农机化, 2005, 26(2):35−38. CHEN D R, WANG S M, WANG X. Status and prospect for recognition technology of farm weeds [J]. Chinese Agriculture Mechanization, 2005, 26(2): 35−38. (in Chinese)
[6]	何义川, 汤智辉, 李光新, 等. 葡萄园除草技术研究现状与发展趋势 [J]. 中国农机化学报, 2018, 39(9):34−37. HE Y C, TANG Z H, LI G X, et al. Research on current status and developing tendency of the vineyard weeding [J]. Journal of Chinese Agricultural Mechanization, 2018, 39(9): 34−37. (in Chinese)
[7]	李东升, 张莲洁, 盖志武, 等. 国内外除草技术研究现状 [J]. 森林工程, 2002, 18(1):17−18. LI D S, ZHANG L J, GAI Z W, et al. Research situations of weeding techniques in abroad and home [J]. Forest Engineering, 2002, 18(1): 17−18. (in Chinese)
[8]	洪晓玮, 陈勇, 杨超淞, 等. 有机蔬菜大棚除草机器人研制 [J]. 制造业自动化, 2021, 43(5):33−36,71. HONG X W, CHEN Y, YANG C S, et al. Development of a weeding robot for organic vegetable greenhouse [J]. Manufacturing Automation, 2021, 43(5): 33−36,71. (in Chinese)
[9]	HASANUZZAMAN M, MOHSIN S M, BHUYAN M H M B, et al. Phytotoxicity, environmental and health hazards of herbicides: Challenges and ways forward[M]//Agrochemicals Detection, Treatment and Remediation. Amsterdam: Elsevier, 2020: 55-99.
[10]	何荣昌. 浅析农田除草剂对土壤生态环境的影响 [J]. 南方农业, 2019, 13(6):187−188. HE R C. Analysis on the influence of herbicide on soil ecological environment in farmland [J]. South China Agriculture, 2019, 13(6): 187−188. (in Chinese)
[11]	东辉, 陈鑫凯, 孙浩, 等. 基于改进YOLOv4和图像处理的蔬菜田杂草检测 [J]. 图学学报, 2022, 43(4):559−569. DONG H, CHEN X K, SUN H, et al. Weed detection in vegetable field based on improved YOLOv4 and image processing [J]. Journal of Graphics, 2022, 43(4): 559−569. (in Chinese)
[12]	兰天, 李端玲, 张忠海, 等. 智能农业除草机器人研究现状与趋势分析 [J]. 计算机测量与控制, 2021, 29(5):1−7. LAN T, LI D L, ZHANG Z H, et al. Analysis on research status and trend of intelligent agricultural weeding robot [J]. Computer Measurement & Control, 2021, 29(5): 1−7. (in Chinese)
[13]	马娟, 董金皋. 微生物除草剂与生物安全 [J]. 植物保护, 2006, 32(1):9−12. MA J, DONG J G. Microbial herbicides and biosafety [J]. Plant Protection, 2006, 32(1): 9−12. (in Chinese)
[14]	金小俊, 孙艳霞, 陈勇, 等. 基于深度学习的草坪杂草识别与除草剂喷施区域检测方法 [J]. 草地学报, 2022, 30(6):1543−1549. JIN X J, SUN Y X, CHEN Y, et al. Weed recognition and herbicide spraying area detection in turf based on deep learning [J]. Acta Agrestia Sinica, 2022, 30(6): 1543−1549. (in Chinese)
[15]	孙艳霞, 陈燕飞, 金小俊, 等. 基于人工智能的青菜幼苗与杂草识别方法 [J]. 福建农业学报, 2021, 36(12):1484−1490. SUN Y X, CHEN Y F, JIN X J, et al. AI differentiation of Bok choy seedlings from weeds [J]. Fujian Journal of Agricultural Sciences, 2021, 36(12): 1484−1490. (in Chinese)
[16]	朱伟兴, 金飞剑, 谈蓉蓉. 基于颜色特征与多层同质性分割算法的麦田杂草识别[J]. 农业机械学报, 2007, 38(12): 120−124. ZHU W X, JIN F J, TAN R R. Weed recognition method based on color feature and hierarchical homogeneity segmentation in wheat field[J]. Transactions of the Chinese Society for Agricultural Machinery, 2007, 38(12): 120−124. (in Chinese)
[17]	BAKHSHIPOUR A, JAFARI A, NASSIRI S M, et al. Weed segmentation using texture features extracted from wavelet sub-images [J]. Biosystems Engineering, 2017, 157: 1−12. doi: 10.1016/j.biosystemseng.2017.02.002
[18]	杨涛, 李晓晓. 机器视觉技术在现代农业生产中的研究进展 [J]. 中国农机化学报, 2021, 42(3):171−181. YANG T, LI X X. Research progress of machine vision technology in modern agricultural production [J]. Journal of Chinese Agricultural Mechanization, 2021, 42(3): 171−181. (in Chinese)
[19]	赵娜, 赵平, 高轶军. 机器视觉技术在我国现代农业生产中的应用研究 [J]. 天津农学院学报, 2015, 22(2):55−58. ZHAO N, ZHAO P, GAO Y J. Study on application of machine vision technology to modern agriculture in China [J]. Journal of Tianjin Agricultural University, 2015, 22(2): 55−58. (in Chinese)
[20]	刘现, 郑回勇, 施能强, 等. 人工智能在农业生产中的应用进展 [J]. 福建农业学报, 2013, 28(6):609−614. LIU X, ZHENG H Y, SHI N Q, et al. Artificial intelligence in agricultural applications [J]. Fujian Journal of Agricultural Sciences, 2013, 28(6): 609−614. (in Chinese)
[21]	OSORIO K, PUERTO A, PEDRAZA C, et al. A deep learning approach for weed detection in lettuce crops using multispectral images [J]. AgriEngineering, 2020, 2(3): 471−488. doi: 10.3390/agriengineering2030032
[22]	彭文, 兰玉彬, 岳学军, 等. 基于深度卷积神经网络的水稻田杂草识别研究 [J]. 华南农业大学学报, 2020, 41(6):75−81. doi: 10.7671/j.issn.1001-411X.202007029 PENG W, LAN Y B, YUE X J, et al. Research on paddy weed recognition based on deep convolutional neural network [J]. Journal of South China Agricultural University, 2020, 41(6): 75−81. (in Chinese) doi: 10.7671/j.issn.1001-411X.202007029
[23]	YU J L, SCHUMANN A W, SHARPE S M, et al. Detection of grassy weeds in bermudagrass with deep convolutional neural networks [J]. Weed Science, 2020, 68(5): 545−552. doi: 10.1017/wsc.2020.46
[24]	YU J L, SHARPE S M, SCHUMANN A W, et al. Deep learning for image-based weed detection in turfgrass [J]. European Journal of Agronomy, 2019, 104: 78−84. doi: 10.1016/j.eja.2019.01.004
[25]	YU J L, SCHUMANN A W, CAO Z, et al. Weed detection in perennial ryegrass with deep learning convolutional neural network [J]. Frontiers in Plant Science, 2019, 10: 1422. doi: 10.3389/fpls.2019.01422
[26]	金小俊, 孙艳霞, 于佳琳, 等. 基于深度学习与图像处理的蔬菜苗期杂草识别方法 [J]. 吉林大学学报(工学版), 2023, 53(8):2421−2429. JIN X J, SUN Y X, YU J L, et al. Weed recognition in vegetable at seedling stage based on deep learning and image processing [J]. Journal of Jilin University:Engineering and Technology Edition, 2023, 53(8): 2421−2429.
[27]	JIN X J, SUN Y X, CHE J, et al. A novel deep learning-based method for detection of weeds in vegetables [J]. Pest Management Science, 2022, 78(5): 1861−1869. doi: 10.1002/ps.6804
[28]	JIN X J, CHE J, CHEN Y. Weed identification using deep learning and image processing in vegetable plantation [J]. IEEE Access, 2021, 9: 10940−10950. doi: 10.1109/ACCESS.2021.3050296
[29]	毛文华, 姜红花, 胡小安, 等. 基于位置特征的行间杂草识别方法 [J]. 农业机械学报, 2007, 38(11):74−76,83. doi: 10.3969/j.issn.1000-1298.2007.11.018 MAO W H, JIANG H H, HU X A, et al. Between-row weed detection method based on position feature [J]. Transactions of the Chinese Society for Agricultural Machinery, 2007, 38(11): 74−76,83. (in Chinese) doi: 10.3969/j.issn.1000-1298.2007.11.018
[30]	PYTORCH. Tensors and dynamic neural networks in python with strong GPU acceleration. [DB/OL]. (2020-01-18)[2020-03-05]. https://github.com/pytorch/pytorch.
[31]	舒娜, 刘波, 林伟伟, 等. 分布式机器学习平台与算法综述 [J]. 计算机科学, 2019, 46(3):9−18. doi: 10.11896/j.issn.1002-137X.2019.03.002 SHU N, LIU B, LIN W W, et al. Survey of distributed machine learning platforms and algorithms [J]. Computer Science, 2019, 46(3): 9−18. (in Chinese) doi: 10.11896/j.issn.1002-137X.2019.03.002
[32]	黄海松, 陈星燃, 韩正功, 等. 基于多尺度注意力机制和知识蒸馏的茶叶嫩芽分级方法 [J]. 农业机械学报, 2022, 53(9):399−407,458. HUANG H S, CHEN X R, HAN Z G, et al. Tea buds grading method based on multiscale attention mechanism and knowledge distillation [J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(9): 399−407,458. (in Chinese)
[33]	李子茂, 徐杰, 郑禄, 等. 基于改进DenseNet的茶叶病害小样本识别方法 [J]. 农业工程学报, 2022, 38(10):182−190. LI Z M, XU J, ZHENG L, et al. Small sample recognition method of tea disease based on improved DenseNet [J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(10): 182−190. (in Chinese)
[34]	GAO S H, CHENG M M, ZHAO K, et al. Res2Net: A new multi-scale backbone architecture [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652−662. doi: 10.1109/TPAMI.2019.2938758
[35]	吕梦棋, 张芮祥, 贾浩, 等. 基于改进ResNet玉米种子分类方法研究 [J]. 中国农机化学报, 2021, 42(4):92−98. LÜ M Q, ZHANG R X, JIA H, et al. Research on seed classification based on improved ResNet [J]. Journal of Chinese Agricultural Mechanization, 2021, 42(4): 92−98. (in Chinese)
[36]	HANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT. IEEE, 2018: 6848-6856.