基于深度學習的森林蟲害無人機實時監(jiān)測方法

孫 鈺，周 焱，袁明帥，劉文萍，駱有慶，宗世祥

?

基于深度學習的森林蟲害無人機實時監(jiān)測方法

孫 鈺1，周 焱1，袁明帥1，劉文萍1※，駱有慶2，宗世祥2

（1. 北京林業(yè)大學信息學院，北京 100083；2. 北京林業(yè)大學林學院，北京 100083）

無人機遙感是監(jiān)測森林蟲害的先進技術，但航片識別的實時性尚不能快速定位蟲害爆發(fā)中心、追蹤災情發(fā)生發(fā)展。該文針對受紅脂大小蠹危害的油松林，使用基于深度學習的目標檢測技術，提出一種無人機實時監(jiān)測方法。該方法訓練精簡的SSD300目標檢測框架，無需校正拼接，直接識別無人機航片。改進的框架使用深度可分離卷積網(wǎng)絡作為基礎特征提取器，針對航片中目標尺寸刪減預測模塊，優(yōu)化默認框的寬高比，降低模型的參數(shù)量和運算量，加快檢測速度。試驗選出的最優(yōu)模型，測試平均查準率可達97.22%，在移動圖形工作站圖形處理器加速下，單張航片檢測時間即可縮短至0.46 s。該方法簡化了無人機航片的檢測流程，可實現(xiàn)受害油松的實時檢測和計數(shù)，提升森林蟲害早期預警能力。

無人機；監(jiān)測；蟲害；目標檢測；深度學習

0 引 言

森林資源在維持生態(tài)平衡、促進經(jīng)濟發(fā)展中發(fā)揮著重要作用。在過去十年中，中國主要林業(yè)有害生物年均發(fā)生面積超過千萬公頃，年均直接經(jīng)濟損失超過千億元[1-2]。如何對受蟲害區(qū)域定位及對危害程度快速準確地監(jiān)測預警，成為多年來國內外林業(yè)專家研究的重要課題。

隨著計算機技術的不斷發(fā)展，結合計算機分析的現(xiàn)代遙感技術在森林監(jiān)測中得到廣泛應用。在航空航天遙感監(jiān)測領域，從衛(wèi)星圖像或高光譜圖像中提取歸一化植被指數(shù)[3-4]，被普遍用來衡量森林健康程度；此外，結合圖像分析[5]、模式識別[6-7]及機器學習[8-10]等計算機算法，實現(xiàn)森林蟲害監(jiān)測、受災害等級分類和統(tǒng)計森林死亡率等，成為研究學者的研究重點。

相較于高空遙感技術，無人機作為一種低空遙感工具，具有操作靈活、拍攝時間自由、飛行成本低等優(yōu)點[11-13]，因此無人機遙感成為森林蟲害監(jiān)測的研究熱點。無人機遙感使用深度學習技術，已實現(xiàn)高精度的樹木及雜草識別[14-16]和地類[17-19]識別，但尚未應用于森林蟲害識別。目前，森林蟲害航片的主要識別方法包括：從無人機采集的彩色圖像中提取顏色特征[20]，對森林樹冠密度、分布等信息進行統(tǒng)計分析；通過計算機圖像分析技術中的圖像分割算法[21-23]提取無人機航拍圖像中的蟲害區(qū)域；利用面向對象的分類算法[24]，以及最鄰近、支持向量機和隨機森林等機器學習分類算法[25-27]，對無人機高光譜圖像中植被品種和受災害等級分類。

應用上述計算機技術實現(xiàn)森林蟲害監(jiān)測，需經(jīng)巡航拍攝、校正拼接或邊緣剔除等預處理和圖像識別3步。校正拼接和圖像識別需無人機降落后，在第二現(xiàn)場使用工作站及專業(yè)軟件完成，用時遠大于飛行時間，極大地延長了監(jiān)測周期，導致機組每次外業(yè)的針對性差，飛行架次及監(jiān)測成本居高不下。無人機監(jiān)測紅脂大小蠹蟲害的窗口期僅為每年6月下旬－9月上旬，期間新一代幼蟲開始危害油松林?，F(xiàn)有方法的實時性不足以在較短的窗口期使用有限的無人機覆蓋廣闊的油松林，難以及時定位蟲害爆發(fā)中心，無法追蹤災情發(fā)生發(fā)展。另外，基于圖像分析技術的無人機監(jiān)測方法要求航拍光影條件一致，監(jiān)測精度依賴預處理。因此，已有的無人機森林蟲害監(jiān)測方法存在監(jiān)測效率低、外業(yè)成本高、依賴預處理等問題，制約了無人機遙感在森林蟲害早期預警的實際應用。

現(xiàn)階段的目標檢測框架中，SSD（single shot multibox detector）[28]作為一種輕量級目標檢測框架，具有可實時、準確率高2個優(yōu)點。SSD300框架在COCO[29]數(shù)據(jù)集測試，mAP（mean average precision）[30]達到41.2%，與基于區(qū)域候選的重量級目標檢測框架Faster R-CNN[31]相當。同時使用NVIDIA TITAN X顯卡，檢測速度達到59 fps，遠快于Faster R-CNN的7 fps，甚至比實時目標檢測框架YOLO[32]的21 fps更快。

本文以受紅脂大小蠹危害的油松作為研究對象，針對無人機監(jiān)測森林蟲害的挑戰(zhàn)，進一步精簡基于深度學習的SSD300目標檢測框架，在林區(qū)無人機遙控現(xiàn)場架設移動圖形工作站，實時監(jiān)測受害油松數(shù)量和分布。

1 數(shù)據(jù)材料

1.1 研究區(qū)域概況

本文試驗點位于遼寧省凌源市，如圖1所示。該試驗點樣地共6塊，每塊樣地大小為30 m×30 m，樣地坡度約為30°，主要樹種為油松。在該地區(qū)，紅脂大小蠹是需要重點監(jiān)測的蟲害之一。

圖1 樣地位置

1.2 數(shù)據(jù)采集

本文以受紅脂大小蠹危害的油松作為測試對象。數(shù)據(jù)采集時間為2017年8月，使用大疆“悟”系列第二代四旋翼航拍機，搭載大疆X5S云臺相機，詳細參數(shù)見表1。無人機掛載鏡頭為奧林巴斯25 mm F1.8定焦鏡頭，飛行高度為50～75 m，掃描拍攝1～6號樣地，航片為含有地理坐標、飛行高度等元信息的JPEG格式圖像，圖像分辨率為5 280×3 956像素。

表1 無人機與大疆 X5S云臺相機主要參數(shù)

1.3 數(shù)據(jù)集建立

本文劃分1、3～6號共5塊樣地的航片作為訓練集，2號樣地的航片作為測試集。其中，1號樣地航片23張，4號樣地11張，其余樣地均為12張。數(shù)據(jù)集共82張圖像，其中訓練集圖像70張，測試集圖像12張。圖像標注通過標注工具完成，標注內容為包圍受害油松邊界的矩形框坐標及分類信息。標注由采集人員完成后，再經(jīng)林學專家復核，部分有異議的圖像結合人工地面調查確認是否為受害油松。

2 檢測方法

2.1 無人機實時監(jiān)測系統(tǒng)

森林蟲害無人機實時監(jiān)測方法主要由航拍無人機、Android無人機遙控器和移動圖形工作站3部分組成，監(jiān)測過程如圖2所示：首先無人機進行定點飛行，對受蟲害林區(qū)進行掃描，間隔拍攝一張分辨率為5 280×3 956像素的林區(qū)航片；無人機遙控器的Android客戶端實時接收并存儲航拍圖像，無需正射校正和拼接，經(jīng)縮小及裁剪后，將12張300×300像素的圖像作為一個批次，通過Tensorflow Serving系統(tǒng)[33]，向移動圖形工作站請求受害油松的檢測識別服務；移動圖形工作站運行精簡的SSD300模型，在圖形處理器（graphics processing unit，GPU）的并行加速下批量完成該批次的受害油松檢測。

圖2 森林蟲害無人機實時監(jiān)測系統(tǒng)架構

2.2 SSD目標檢測框架

SSD目標檢測框架是使用深度神經(jīng)網(wǎng)絡作為特征提取器的輕量級一階段目標檢測方法[28]。如圖3a所示，文獻[22]中SSD300框架使用VGG16[34]作為基礎特征提取器，并在VGG16末尾增加抽象程度更高的特征層，最終以多尺度特征圖P1～P6上的默認框為錨點，預測目標的類別及位置。P1～P6的每個單元都與一組默認框相關聯(lián)，每組默認框在一個正方形基礎框上，覆蓋寬高比為{2，1/2，3，1/3}的默認框。用基礎框與輸入圖像的面積比作為該組默認框的基礎比例，各層基礎比例分別為{0.1，0.2，0.37，0.54，0.71，0.88}。圖3b為不同尺度特征圖生成的基礎框示例。圖3c和圖3d為特征圖P2和P3生成的一組默認框示例，白色框為受害油松的標注框，黑色網(wǎng)格表示特征圖P2、P3的單元數(shù)目，分別為19×19和10×10。虛線框表示以單元紅色中心點為基準生成的一組默認框，其中藍色框為基礎框，黃色框為其他寬高比的默認框。P2與P3生成的默認框與標注框的IoU（intersection over union）最高。

SSD的損失函數(shù)由位置損失（L）和分類損失（L）組成，定義為

注：圖3a中Conv表示卷積層，參數(shù)s1、s2表示卷積步長分別1和2。圖3c和圖3d中，白色框為受害油松的標注框，黑色網(wǎng)格表示特征圖P2、P3的單元數(shù)目。虛線框表示以單元紅色中心點為基準生成的一組默認框，其中藍色框為基礎框，黃色框為其他寬高比的默認框。

2.3 基于深度可分離卷積結構的受害油松檢測框架

2.4 評價指標

本文采用單張航片的檢測時間及受害油松的測試平均查準率（average precision，AP）[30]作為檢測速度和精確度的評價指標。AP為精確率（precision）和召回率（recall）曲線下的面積，精確率和召回率的定義為

注：圖4a中，dw為深度卷積，pw為點卷積；圖4b中，D表示特征圖及濾波器分辨率大小，M、N均為通道數(shù)。

3 結果與分析

3.1 模型訓練與部署

訓練模型的深度學習服務器安裝為Ubuntu 16.04 LTS 64位系統(tǒng)，采用TensorFlow[37]深度學習開源框架。服務器搭載AMD Ryzen 1700X CPU（64GB內存），并采用NVIDIA TITAN Xp GPU（12GB顯存）。訓練階段采用動量為0.9的隨機梯度下降算法進行優(yōu)化，設置初始學習率為0.001，正則化系數(shù)設為0.000 04，以16張圖像為一個批次，共訓練100 000次，每35 000次學習率下降原來的0.1倍。訓練過程采用的數(shù)據(jù)擴充方式為隨機水平翻轉和隨機圖像裁剪。

訓練完畢后，模型經(jīng)計算圖精簡和常量化后，部署至火影影刃Z5移動圖形工作站。工作站搭載Intel i7-8750H CPU（16GB內存）及NVIDIA GTX 1050Ti GPU（4GB顯存）。

3.2 基礎特征提取器對檢測速度及精確度的影響

基礎特征提取器是影響模型檢測速度的因素之一。表2為不同目標檢測框架的測試時間及測試AP，由表2可知，模型1將深度可分離卷積網(wǎng)絡作為基礎特征提取器，相比基于VGG16的SSD300目標檢測框架，參數(shù)量減少約528 MB，單張圖像檢測時間提高了4 s。如5a的PR曲線所示，模型0的AP為98.70%，模型1比模型0的AP只降低了1.01%?；A特征提取框架的改變對精確度的影響較小，但會大幅提升檢測速度。

3.3 精簡SSD300預測模塊

精簡SSD300目標檢測框架的預測模塊可加快檢測速度。如表2所示，預測模塊保留了P2、P3的模型3、4，相比預測模塊完整的模型1和保留了P2～P4的模型2，參數(shù)量更少，模型檢測時間縮短至0.46 s。而由圖5b可知，模型3的精確度與模型1相比，幾乎沒有降低。模型3和模型4刪減預測模塊各層默認框寬高比后，檢測速度最快，均為0.46 s。如圖5c所示，對比模型3，模型4的AP只降低了0.68%，而參數(shù)量進一步降低至18.8 MB。試驗表明，針對本文數(shù)據(jù)集，模型4只保留SSD300目標檢測框架預測模塊中P2、P3層，以及適合數(shù)據(jù)集檢測目標的默認框寬高比，AP達到97.22%，相比原模型僅降低了1.48%，可在保證精確度的前提下，降低模型參數(shù)量，最大程度地提升檢測速度。

表2 不同檢測框架的單張圖像檢測時間及測試平均查準率

圖5 模型0～4的測試集Precision-Recall曲線

3.4 檢測結果與典型錯誤分析

由表2可知，與重量級的二階段目標檢測框架Faster R-CNN相比，模型4的參數(shù)量僅為Faster R-CNN的10.85%，單張航片檢測時間僅為Faster R-CNN的7.32%，而測試AP相當，僅降低0.69%。選擇分類置信度>0.6為閾值，此時模型4的精確率和召回率分別為98.04%和83.33%。無人機在75 m高度拍攝的圖像地面覆蓋范圍為38.18 m×50.95 m，無人機以經(jīng)濟速度15 m/s飛行，移動到無重疊的下一拍攝點需3.4 s，本系統(tǒng)完成一個批次的檢測僅需0.46 s，可實現(xiàn)對森林蟲害的實時監(jiān)測。

不經(jīng)正射校正，坡地油松航片邊緣為側視，并存在鏡頭畸變。圖6a和圖6b分別為航片中心正射受害油松和航片邊緣側視受害油松的檢測結果，可知本文方法不僅能準確地檢測出受害油松的正射樹冠，對邊緣側視的坡地樹冠也有較強的魯棒性。

本文影響受害油松檢測精確度的主要原因如下：1）松油分布密集情況下，多株油松被歸并而造成檢測框的定位不準，如圖6c所示；2）位于圖像邊緣的不完整受害油松不易檢測而造成漏檢，如圖6d所示。

圖6 典型測試樣本的檢測結果示例

4 結 論

針對傳統(tǒng)航片識別技術監(jiān)測效率低、外業(yè)成本高、依賴預處理等問題，本文使用深度學習技術，提出了一種面向紅脂大小蠹的無人機實時監(jiān)測方法。本文在SSD300目標檢測框架的基礎上，將深度可分離卷積網(wǎng)絡作為基礎特征提取器，預測模塊精簡至P2和P3，且默認框的寬高比只保留{1，2，1/2}。結果表明：無需航片拼接、正射校正及邊緣剔除等預處理，受害油松檢測平均查準率可達97.22%，相比原模型僅降低1.48%，而模型參數(shù)量從550.1降低至18.8 MB，在移動圖形工作站的圖形處理器加速下，單張圖像的檢測時間僅為0.46 s，實現(xiàn)了受害油松的實時無人機監(jiān)測。本方法簡化了無人機航片的檢測流程，可提高機組每次外業(yè)的針對性，及時定位蟲害爆發(fā)中心，追蹤災情發(fā)生發(fā)展，滿足森林蟲害早期預警對時效性的需求。

本文目前僅對受害油松進行單分類檢測，未對油松的受害等級分類，未來將進一步探索細粒度森林蟲害監(jiān)測方法。

[1] 劉萍，梁倩玲，陳夢，等. 林業(yè)有害生物災害損失評估指標體系構建[J]. 林業(yè)科學，2016，52(6)：101－107. Liu Ping, Liang Qianling, Chen Meng，et al. Construction on index system of forest pest disaster loss assessment[J]. Scientia Silvae Sinicae, 2016, 52(6): 101－107. (in Chinese with English abstract)

[2] 史潔青, 馮仲科, 劉金成. 基于無人機遙感影像的高精度森林資源調查系統(tǒng)設計與試驗[J]. 農業(yè)工程學報, 2017, 33(11): 82－90. Shi Jieqing, Feng Zhongke, Liu Jincheng. Design and experiment of high precision forest resource investigation system based on UAV remote sensing images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(11): 82－90. (in Chinese with English abstract)

[3] Gooshbor L, Bavaghar M P, Amanollahi J，et al. Monitoring infestations of oak forests by tortrix viridana (Lepidoptera: Tortricidae) using remote sensing[J]. Plant Protection Science, 2016, 52(4): 270－276.

[4] Olsson P O, Lindstr?m J, Eklundh L. Near real-time monitoring of insect induced defoliation in subalpine birch forests with MODIS derived NDVI[J]. Remote Sensing of Environment, 2016, 181: 42－53.

[5] Patil J K, Kumar R. Advances in image processing for detection of plant diseases[J]. Journal of Advanced Bioinformatics Applications and Research, 2011, 2(2): 135－141.

[6] Liaghat S, Ehsani R, Mansor S，et al. Early detection of basal stem rot disease (Ganoderma) in oil palms based on hyperspectral reflectance data using pattern recognition algorithms[J]. International Journal of Remote Sensing, 2014, 35(10): 3427－3439.

[7] Niteshkpoona, Riyadismail. Discriminating the occurrence of pitch canker fungus in Pinus radiata trees using QuickBird imagery and artificial neural networks[J]. Journal of the South African Forestry Association, 2013, 75(1): 29－40.

[8] 趙玉，王紅，張珍珍. 基于遙感光譜和空間變量隨機森林的黃河三角洲刺槐林健康等級分類[J]. 遙感技術與應用，2016，31(2)：359－367. Zhao Yu, Wang Hong, Zhang Zhenzhen. Forest healthy classification of robinia pseudoacacia in the yellow river delta, China based on spectral and spatial remote sensing variables using random forest[J]. Remote Sensing Technology and Application, 2016, 31(2): 359－367. (in Chinese with English abstract)

[9] Belgiu M, Dr?gu? L. Random forest in remote sensing: A review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114: 24－31.

[10] Omer G, Mutanga O, Elfatih M Abdel-Rahman，et al. Performance of support vector machines and artificial neural network for mapping endangered tree species using worldview-2 data in Dukuduku Forest, South Africa[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2016, 8(10): 4825－4840.

[11] Pajares G. Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs)[J]. Photogrammetric Engineering & Remote Sensing, 2015, 81(4): 281－329.

[12] 付卓新，姚銳. 淺談森林資源調查中無人機遙感的應用[J]. 南方農業(yè)，2015，9(36)：246，248.

[13] Dash J P, Watt S M, Pearse G D，et al. Assessing very high resolution UAV imagery for monitoring forest health during a simulated disease outbreak[J]. International Journal of Photogrammetry & Remote Sensing, 2017, 131: 1－14.

[14] Fan Zhun, Lu Jiewei, Gong Maoguo，et al. Automatic tobacco plant detection in uav images via deep neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2018, 11(3): 876－887.

[15] Ribera J, Chen Y, Christopher Boomsma, et al. Counting plants using deep learning[C]// IEEE Global Conference on Signal and Information Processing. 2017: 1344－1348.

[16] Li Liujun, Fan Youheng, Huang Xiaoyun, et al.Real-time UAV weed scout for selective weed control by adaptive robust control and machine learning algorithm[C]// Orlando: American Society of Agricultural and Biological Engineers, 2016.

[17] 孫鈺，韓京冶，陳志泊，等. 基于深度學習的大棚及地膜農田無人機航拍監(jiān)測方法[J]. 農業(yè)機械學報，2018，49(2)：133－140. Sun Yu, Han Jingye, Chen Zhibo，et al. Monitoring method for uav image of greenhouse and plastic-mulched landcover based on deep learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2018,49(2): 133－140. (in Chinese with English abstract)

[18] 黨宇，張繼賢，鄧喀中，等. 基于深度學習AlexNet的遙感影像地表覆蓋分類評價研究[J]. 地球信息科學學報，2017，19(11)：1530－1537. Dang Yu, Zhang Jixian, Deng Kazhong，et al. Study on the evaluation of land cover classification using remote sensing images based on AlexNet[J]. Journal of Geo-information Science, 2017, 19(11): 1530－1537. (in Chinese with English abstract)

[19] 金永濤, 楊秀峰, 高濤，等. 基于面向對象與深度學習的典型地物提取[J]. 國土資源遙感, 2018, 30(1): 22－29.

Jin Yongtao, Yang Xiufeng, Guo Tao, et al. The typical object extraction method based on object － oriented and deep learning[J]. Remote Sensing for Land and Resources, 2018, 30(1): 22－29. (in Chinese with English abstract)

[20] Chianucci F, Disperati L, Guzzi D，et al. Estimation of canopy attributes in beech forests using true colour digital images from a small fixed-wing UAV[J]. International Journal of Applied Earth Observation & Geoinformation, 2016, 47: 60－68.

[21] Liu Wenping, Fei Yunqiao, Wu Lan. A marker-based watershed algorithm using fractional calculus for unmanned aerial vehicle image segmentation[J]. Journal of Information & Computational Science, 2015, 12(14): 5327－5338.

[22] Zhong Tingyu, Liu Wenping, Luo Youqing，et al. A new type-2 fuzzy algorithm for unmanned aerial vehicle image segmentation[J]. International Journal of Multimedia & Ubiquitous Engineering, 2017, 12(5): 75－90.

[23] 張軍國，馮文釗，胡春鶴，等. 無人機航拍林業(yè)蟲害圖像分割復合梯度分水嶺算法[J]. 農業(yè)工程學報，2017，33(14)：93－99. Zhang Junguo, Feng Wenzhao, Hu Chunhe，et al. Image segmentation method for forestry unmanned aerial vehicle pest monitoring based on composite gradient watershed algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(14): 93－99. (in Chinese with English abstract)

[24] Chenari A, Erfanifard Y, Dehghani M, et al. Woodland mapping at single-tree levels using object-oriented classification of unmanned aerial vehicle (UAV) images[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, 42: 43－49.

[25] Hung C, Zhe X, Sukkarieh S. Feature learning based approach for weed classification using high resolution aerial images from a digital camera mounted on a UAV[J]. Remote Sensing, 2014, 6(12): 12037－12054.

[26] Yuan Huanhuan, Yang Guijun, Li Changchun, et al. Retrieving soybean leaf area index from unmanned aerial vehicle hyperspectral remote sensing: Analysis of RF, ANN, and SVM regression models[J]. Remote Sensing, 2017, 9(4): 309.

[27] Cao Jingjing, Leng Wanchun, Liu Kai, et al. Object-based mangrove species classification using unmanned aerial vehicle hyperspectral images and digital surface models[J]. Remote Sensing, 2018, 10(1): 89.

[28] Liu Wei, Anguelov D, Erhan D, et al. SSD: Single shot multibox detector[C]// European Conference On Computer vision. 2016: 21－37.

[29] Lin Tsung Yi, Maire M, Belongie S, et al. Microsoft COCO: Common objects in context[J]. European Conference on Computer Vision. Springer International Publishing, 2014, 8693: 740－755.

[30] Everingham M, Eslami S M A, Gool L V, et al. The pascal visual object classes challenge: A retrospective[J]. International Journal of Computer Vision, 2015, 111(1): 98－136.

[31] Ren S, He K, Girshick R，et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// International Conference on Neural Information Processing Systems. 2015: 91－99.

[32] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 779－788.

[33] Olston C, Fiedel N, Gorovoy K, et al. Tensorflow-serving: Flexible, high-performance ML serving [C]//31st Conference on Neural Information Processing Systems. 2017.

[34] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition [J/OL]. 2015, https://arxiv.org/abs/1409.1556.

[35] Girshick R. Fast R-CNN[C]// Computer Vision (ICCV), 2015 IEEE International Conference on. IEEE, 2015: 1440－1448.

[36] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications [J/OL]. 2017, https://arxiv.org/abs/1704.04861.

[37] Abadi M, Barham P, Chen J, et al. TensorFlow: A System for Large-Scale Machine Learning[C]// 12th USENIX Symposium on Operating Systems Design and Implementation, 2016: 265－283.

UAV real-time monitoring for forest pest based on deep learning

Sun Yu1, Zhou Yan1, Yuan Mingshuai1, Liu Wenping1※, Luo Youqing2, Zong Shixiang2

(1.100083,2.100083,)

The unmanned aerial vehicle (UAV) remote sensing featured by low cost and flexibility offers a promising solution for pests monitoring by acquiring high resolution forest imagery. So the forest pest monitoring system based on UAV is essential to the early warning of red turpentine beetle (RTB) outbreaks. However, the UAV monitoring method based on image analysis technology suffers from inefficiency and depending on pre-processing, which prohibits the practical application of UAV remote sensing. Due to the long process flow, traditional methods can not locate the outbreak center and track the development of epidemic in time. The RTB is a major forestry invasive pest which damages the coniferous species of pine trees in northern China. This paper focuses on the detection of pines infected by RTBs. A real-time forest pest monitoring method based on deep learning is proposed for UAV forest imagery. The proposed method was consisted of three steps: 1) The UAV equipped with prime lens camera scans the infected forest and collects images at fixes points. 2) The Android client on UAV remote controller receives images and then requests the mobile graphics workstation for infected trees detection through TensorFlow Serving in real time. 3) The mobile graphics workstation runs a tailored SSD300 (single shot multibox detector) model with graphics processing unit (GPU) parallel acceleration to detect infected trees without orthorectification and image mosaic. Compared with Faster R-CNN and other two-stage object detection frameworks, SSD, as a lightweight object detection framework, shows the advantages of real-time and high accuracy. The original SSD300 object detection framework uses truncated VGG16 as basic feature extractor and the 6 layers (named P1-P6) prediction module to detect objects with different sizes. The proposed tailored SSD300 object detection framework includes two parts. First, a 13-layer depthwise separable convolution is used as basic feature extractor, which reduces several times computation overhead compared with the standard convolutions in VGG16. Second, most loss is derived from positive default boxes and these boxes mainly concentrated in P2 and P3 due to the constraints of crown size, UAV flying height and lens’ focal length. Therefore, the tailored SSD300 retains only P2 and P3 as prediction module and the other prediction layers are deleted to further reduce computation overhead. Besides, aspect ratio of default boxes is set to {1, 2, 1/2}, since the aspect ratio of crown is approximate 1. The UAV imagery is collected on 6 experimental plots at 50-75 m height. The photos of No.2 experimental plot are considered as test set and the rest are train set. A total of 82 aerial photos are used in the experiment, including 70 photos in the train set and 12 photos in the test set. The AP and run time of five models are evaluated. The average precision (AP) of the tailored SSD300 model reaches up to 97.22%, which is lower than the AP of original SSD300. While the proposed model has only 18.8 MB parameters, reducing above 530 MB compared with the original model. And the run time is 0.46 s on a mobile workstation equipped with NVIDIA GTX 1050Ti GPU, while the original model needs 4.56 s. Experimental results demonstrate that the downsize of basic feature extractor and prediction module speed up detection with a little impact on AP. The maximum coverage of aerial photo captured at 75 m height is 38.18 m×50.95 m. When the UAV has a horizontal speed of 15 m/s, it takes 3.4 s to move to the next shooting point without overlap, longer than the detection time. Therefore, the proposed method can simplify the detection process of UAV monitoring and realizes the real-time detection of RTB damaged pines, which introduces a practical and applicable solution for early warning of RTB outbreaks.

unmanned aerial vehicle; monitoring; diseases; object detection; deep learning

10.11975/j.issn.1002-6819.2018.21.009

TP391.41

A

1002-6819(2018)-21-0074-08

2018-06-13

2018-09-10

北京市科技計劃“影響北京生態(tài)安全的重大鉆蛀性害蟲防控技術研究與示范”（Z171100001417005）

孫 鈺，副教授，主要從事林業(yè)物聯(lián)網(wǎng)與人工智能研究。 Email：sunyv@bjfu.edu.cn

劉文萍，教授，博士生導師，主要從事計算機圖像及視頻分析與處理、模式識別與人工智能研究。Email：wendyl@vip.163.com

孫 鈺，周 焱，袁明帥，劉文萍，駱有慶，宗世祥. 基于深度學習的森林蟲害無人機實時監(jiān)測方法[J]. 農業(yè)工程學報，2018，34(21)：74－81. doi：10.11975/j.issn.1002-6819.2018.21.009 http://www.tcsae.org

Sun Yu, Zhou Yan, Yuan Mingshuai, Liu Wenping, Luo Youqing, Zong Shixiang. UAV real-time monitoring for forest pest based on deep learning[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(21): 74－81. (in Chinese with English abstract) doi：10.11975/j.issn.1002-6819.2018.21.009 http://www.tcsae.org