基于深度学习和无人机遥感技术的玉米雄穗检测研究

梁胤豪; 陈全; 董彩霞; 杨长才

doi:10.19303/j.issn.1008-0384.2020.04.014

基于深度学习和无人机遥感技术的玉米雄穗检测研究

doi: 10.19303/j.issn.1008-0384.2020.04.014

梁胤豪^{1, 2,},
陈全^{2, 3},
董彩霞^{2, 3},
杨长才^{2, 3, 4, ,}

1.
西北大学数学学院，陕西　西安　710127
2.
福建农林大学农林大数据研究院，福建　福州　350002
3.
福建农林大学计算机与信息学院/数字福建农林大数据研究所，福建　福州　350002
4.
智慧农林福建省高校重点实验室，福建　福州　350002

基金项目: 国家自然科学基金项目（61802064、61701117）；福建省自然科学基金项目（2019J01402）

详细信息

作者简介:
梁胤豪（1998−），男，研究方向：计算机应用（E-mail：lyhmath@163.com）

通讯作者:
杨长才（1981−），男，博士，副教授，研究方向：计算机视觉与作物表型识别（E-mail：changcaiyang@gmail.com）

中图分类号: S 513；S 127
计量
- 文章访问数: 1433
- HTML全文浏览量: 691
- PDF下载量: 66
- 被引次数: 0
出版历程
- 收稿日期: 2020-03-12
- 修回日期: 2020-04-15
- 刊出日期: 2020-04-01

Application of Deep-learning and UAV for Field Surveying Corn Tassel

LIANG Yin-hao^{1, 2
,},
CHEN Quan^{2, 3},
DONG Cai-xia^{2, 3},
YANG Chang-cai^{2, 3, 4
, ,}

1.
School of Mathematics, Northwest University, Xian, Shaanxi 710127, China
2.
Institute of Big Data for Agriculture and Forestry, Fujian Agriculture and Forestry University, Fuzhou, Fujian 350002, China
3.
Institute of Digital Research and Big Data/College of Computer and Information Science, Fujian Agriculture and Forestry University, Fuzhou, Fujian 350002, China
4.
Key Laboratory of Smart Computing, Fujian Agriculture and Forestry University, Fuzhou, Fujian 350002, China

摘要

摘要: 目的玉米雄穗在玉米的生长过程和最终产量中起关键作用，使用无人机采集玉米抽穗期的RGB图像，研究不同的目标检测算法，构建适用于无人机智能检测玉米雄穗的模型，自动计算图像中雄穗的个数。方法使用无人飞行器（UAV）在25 m飞行高度下获得大量玉米抽穗时期的RGB图像，裁剪并标注出图像中玉米雄穗的位置和大小，训练数据和测试数据按照3:1的比例划分数据集；在深度学习框架MXNet下，利用这些数据集，分别训练基于ResNet50的Faster R-CNN、基于ResNet50的SSD、基于mobilenet的SSD和YOLOv3等4种模型，对比4种模型的准确率、检测速度和模型大小。结果使用无人机采集了236张图像，裁剪成1024×1024大小的图片，去除成像质量差的图像，利用标注软件labelme获得100张标注的玉米雄穗数据集；最终得到4个模型的mAP值分别为0.73、0.49、0.58和0.72。在测试数据集上进行测试，Faster R-CNN模型的准确率最高为93.79%，YOLOv3的准确率最低，仅有20.04%，基于ResNet50的SSD和基于mobilenet的SSD分别为89.9%和89.6%。在识别的速度上，SSD_mobilenet最快（8.9 samples·s⁻¹），Faster R-CNN最慢（2.6 samples·s⁻¹），YOLOv3检测速度为3.47 samples·s⁻¹, SDD_ResNet50检测速度为7.4 samples·s⁻¹。在模型大小上，YOLO v3的模型最大，为241 Mb，SSD_mobilenet的模型最小，为55.519 Mb。结论由于无人机的机载平台计算资源稀缺，综合模型的速度、准确率和模型大小考虑，SSD_mobilenet最适于部署在无人机机载系统上用于玉米雄穗的检测。
- 无人机 /
- 目标检测 /
- 玉米 /
- 雄穗
Abstract: Objective Deep-learning and computation were applied to analyze the images collected by drones on the status of tassel on corn plants in the field for estimating crop growth and forecasting production. Method Drones, or unmanned aerial vehicles (UAV), flying at a height of 25 m above corn crop in the field were used to generate RGB images showing the position and size of tassel on the plants at heading stage. Under the deep-learning framework of MXNet, the data sets on a 3-to-1 training-to-testing ratio were fed into 4 models of the ResNet50-based Faster R-CNN, the ResNet50-based SSD, the mobilenet-based SSD, and YOLOv3. The algorithms provided by the models were tested to intelligently extract information from the images for an accurate and rapid report on the status of corn tassel. Results mThe 236 UAV-collected images were cropped individually into 1024×1024 size. Those of poor quality were discarded to result in 100 labeled datasets using the Labelme software. The mAPs of the 4 models were 0.73, 0.49, 0.58 and 0.72, respectively. The highest accuracy rate of 93.79% on the test was obtained from the Faster R-CNN model, followed by 89.9% from SSD_ResNet50, 89.6% from SSD_mobilenet, and 20.04% from YOLOv3. On processing speed, SSD_mobilenet was the fastest at 8.9 samples·s⁻¹, followed by YOLOv3 at 3.47 samples·s⁻¹, SSD_ResNet50 at 7.4 samples·s⁻¹, and Faster R-CNN at 2.6 samples·s⁻¹. Among the 4 models, YOLOv3 was the largest, 241 Mb in size, while SSD_mobilenet the smallest 55.519 Mb. Conclusion Considering the scarcity of available resources on the airborne UAV platform, as well as the detection accuracy, processing speed, and size of the programs, the SSD_mobilenet model was selected as the choice for the field surveying of corn tassel by UAV.
- UAV /
- object detection /
- corn /
- tassel

HTML全文

图 1 无人机航线规划

Figure 1. Planning of UAV flight routes

下载: 全尺寸图片幻灯片

图 2 试验田、航线设置、DJI无人机与LabelImg数据标注

Figure 2. Experimentation field, flight routes, DJI drone, and LabelImg data annotation

下载: 全尺寸图片幻灯片

图 3 模型在训练过程中的损失函数曲线

Figure 3. Loss function of model during training

下载: 全尺寸图片幻灯片

图 4 SSD_mobilenet的预测结果

Figure 4. Prediction by SSD-mobilenet

下载: 全尺寸图片幻灯片

图 5 SSD_ResNet50的预测结果

Figure 5. Prediction by SSD_ResNet50

下载: 全尺寸图片幻灯片

图 6 Faster R-CNN的预测结果

Figure 6. Prediction by Faster R-CNN

下载: 全尺寸图片幻灯片

图 7 YOLO v3的预测结果

Figure 7. Prediction by YOLOv3

下载: 全尺寸图片幻灯片

图 8 模型的预测结果对比

注：Faster R-CNN、YOLO v3、SSD_ResNet50和SSD_mobilenet的计数准确率分别为93.79%、20.04%、87.6%和89.9%.

Figure 8. Comparison of predictions by various models

Note: Detection accuracy of Faster R-CNN was 93.79%; YOLO v3, 20.04%; SSD_ResNet50, 87.6%; and, SSD_mobilenet, 89.9%.

下载: 全尺寸图片幻灯片

图 9 模型训练过程的mAP曲线

Figure 9. mAP curves of models in testing

下载: 全尺寸图片幻灯片

表 1 试验硬件与软件信息

Table 1. Information on hardware and software for testing

硬件信息 Hardware information			软件信息 Software information
平台 Platform	型号 Model	参数 Parameters	平台 Platform	版本 Version
CPU	E5-2680v2	2.8 GHz	CUDA	10.0
RAM	DDR3	128 G	CUDNN	7.6.5
GPU	1080ti	11 G	MXNet	1.5.0

下载: 导出CSV

表 2 模型训练的超参数

Table 2. Hyperparameters for model training

参数 Parameters	Faster R-CNN	YOLO v3	SSD	SSD
base-network	ResNet50	darknet53	ResNet50	mobilenet
batch-size	4	8	16	16
epochs	400	300	300	400
learning rate	0.001	0.0001	0.0001	0.0001

下载: 导出CSV

表 3 模型的mAP

Table 3. mAPs of models

模型 Model	Faster R-CNN	SSD_ResNet50	SSD_mobilenet	YOLO v3
mAP	0.7306	0.4905	0.5780	0.7265

下载: 导出CSV

表 4 模型的测试误差和计数准确率比较

Table 4. Comparison of test errors and detection accuracies by models

模型 Model	Faster R-CNN	SSD_ResNet50	SSD_mobilenet	YOLO v3
误差均值 Mean error	4.7308	9.2692	7.5385	62.1154
均方差 Mean square error	5.3649	11.5175	8.8915	14.0311
计算准确率 Calculation accuracy/%	93.79	87.60	89.90	20.04

下载: 导出CSV

表 5 模型的处理速度和参数大小比较

Table 5. Comparison of processing speeds and parameters of models

模型 Model	Faster R-CNN	SSD_ResNet50	SSD_mobilenet	YOLO v3
处理速度 Processing speed/（samples·s⁻¹）	2.6	7.4	8.9	3.47
参数大小 Parameter size/M	133.873	144.277	55.519	241.343

下载: 导出CSV

参考文献(14)

[1]	HUANG J X, GÓMEZ-DANS J L, HUANG H, et al. Assimilation of remote sensing into crop growth models: Current status and perspectives [J]. Agricultural and Forest Meteorology, 2019, 276/277: 107609. doi: 10.1016/j.agrformet.2019.06.008
[2]	MADEC S, JIN X L, LU H, et al. Ear density estimation from high resolution RGB imagery using deep learning technique [J]. Agricultural and Forest Meteorology, 2019, 264: 225−234. doi: 10.1016/j.agrformet.2018.10.013
[3]	QUAN L Z, FENG H Q, LV Y, et al. Maize seedling detection under different growth stages and complex field environments based on an improved Faster R–CNN [J]. Biosystems Engineering, 2019, 184: 1−23. doi: 10.1016/j.biosystemseng.2019.05.002
[4]	ZHOU C Q, LIANG D, YANG X D, et al. Recognition of wheat spike from field based phenotype platform using multi-sensor Fusion and improved maximum entropy segmentation algorithms [J]. Remote Sensing, 2018, 10(2): 246. doi: 10.3390/rs10020246
[5]	KURTULMUŞ F, KAVDIR İ. Detecting corn tassels using computer vision and support vector machines [J]. Expert Systems with Applications, 2014, 41(16): 7390−7397. doi: 10.1016/j.eswa.2014.06.013
[6]	LU H, CAO Z G, XIAO Y, et al. Fine-grained maize tassel trait characterization with multi-view representations [J]. Computers and Electronics in Agriculture, 2015, 118: 143−158. doi: 10.1016/j.compag.2015.08.027
[7]	LU H, CAO Z G, XIAO Y, et al. TasselNet: counting maize tassels in the wild via local counts regression network [J]. Plant Methods, 2017, 13: 79. doi: 10.1186/s13007-017-0224-0
[8]	LIU Y L, CEN C J, CHE Y P, et al. Detection of maize tassels from UAV RGB imagery with faster R-CNN [J]. Remote Sensing, 2020, 12(2): 338. doi: 10.3390/rs12020338
[9]	LabelImg (Available online) [DB/OL]. https://www.github.com/tzutalin/labelImg (accessed on 25 December 2015).
[10]	LIU L, OUYANG W, WANG X G, et al. Deep learning for generic object detection: a survey [J]. International Journal of Computer Vision, 2020, 128(2): 261−318. doi: 10.1007/s11263-019-01247-4
[11]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[M]. Computer Vision-ECCV. Cham: Springer International Publishing, 2016: 21－37.[LinkOut]
[12]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017. Honolulu, HI. IEEE, 2017.
[13]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149. doi: 10.1109/TPAMI.2016.2577031
[14]	CHEN T Q, LI M, LI Y T, et al. MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems[EB/OL]. 2015-12-03. https://arxiv.org/abs/1512.01274.