Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV)

English version

Autoři: Bipul Neupane ^aff001; Teerayut Horanont ^aff001; Nguyen Duy Hung ^aff001
Působiště autorů: School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Pathum Thani, Thailand ^aff001
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0223906

Souhrn

The production of banana—one of the highly consumed fruits—is highly affected due to loss of certain number of banana plants in an early phase of vegetation. This affects the ability of farmers to forecast and estimate the production of banana. In this paper, we propose a deep learning (DL) based method to precisely detect and count banana plants on a farm exclusive of other plants, using high resolution RGB aerial images collected from Unmanned Aerial Vehicle (UAV). An attempt to detect the plants on the normal RGB images resulted less than 78.8% recall for our sample images of a commercial banana farm in Thailand. To improve this result, we use three image processing methods—Linear Contrast Stretch, Synthetic Color Transform and Triangular Greenness Index—to enhance the vegetative properties of orthomosaic, generating multiple variants of orthomosaic. Then we separately train a parameter-optimized Convolutional Neural Network (CNN) on manually interpreted banana plant samples seen on each image variants, to produce multiple results of detection on our region of interest. 96.4%, 85.1% and 75.8% of plants were correctly detected on three of our dataset collected from multiple altitude of 40, 50 and 60 meters, of same farm. Further discussion on results obtained from combination of multiple altitude variants are also discussed later in the research, in an attempt to find better altitude combination for data collection from UAV for the detection of banana plants. The results showed that merging the detection results of 40 and 50 meter dataset could detect the plants missed by each other, increasing recall upto 99%.

Klíčová slova:

Grasses – Algorithms – Imaging techniques – Leaves – Trees – Image processing – Farms – Bananas

Zdroje

1. Dale JL. Banana bunchy top: An economically important tropical plant virus disease. Advances in virus research. 1987;33:301–326. doi: 10.1016/S0065-3527(08)60321-8 3296696

2. STEWART L, CAMPAGNOLO D, DANIELLS J, LEMIN C, GOEBEL R, PINESE B, et al. Tropical banana information kit. Nambour: Queensland Department of Primary Industries. 1998.

3. Bastiaanssen WG, Molden DJ, Makin IW. Remote sensing for irrigated agriculture: examples from research and possible applications. Agricultural water management. 2000;46(2):137–155. doi: 10.1016/S0378-3774(00)00080-9

4. Mulla DJ. Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps. Biosystems engineering. 2013;114(4):358–371. doi: 10.1016/j.biosystemseng.2012.08.009

5. Zhang C, Kovacs JM. The application of small unmanned aerial systems for precision agriculture: a review. Precision agriculture. 2012;13(6):693–712. doi: 10.1007/s11119-012-9274-5

6. Turner D. Banana plant growth. 1. Gross morphology. Australian Journal of Experimental Agriculture. 1972;12(55):209–215. doi: 10.1071/EA9720209

7. Ke Y, Quackenbush LJ. A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. International Journal of Remote Sensing. 2011;32(17):4725–4747. doi: 10.1080/01431161.2010.494184

8. Singh K, Sohlberg S, Sokolov V, et al. Conceptual framework for the selection of appropriate remote sensing techniques. 1986.

9. Pinz A. Tree isolation and species classification. In: Proceedings of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry, Victoria, BC; 1998. p. 127–139.

10. Gougeon FA, Leckie DG, et al. Forest regeneration: Individual tree crown detection techniques for density and stocking assessment. In: Proceedings of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry. Canadian Forest Service, Pacific Forestry Centre Victoria, BC; 1998. p. 10–12.

11. Brandtberg T. Automatic individual tree based analysis of high spatial resolution aerial images on naturally regenerated boreal forests. Canadian Journal of Forest Research. 1999;29(10):1464–1478. doi: 10.1139/x99-150

12. Pollock R. Individual tree recognition based on a synthetic tree crown image model. In: Proc. of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry. Victoria, British Columbia, Canada; 1998. p. 25–34.

13. Johansen K, Sohlbach M, Sullivan B, Stringer S, Peasley D, Phinn S. Mapping banana plants from high spatial resolution orthophotos to facilitate plant health assessment. Remote Sensing. 2014;6(9):8261–8286. doi: 10.3390/rs6098261

14. She T, Ehsani R, Robbins J, Leiva JN, Owen J. Applications of small UAV systems for tree and nursery inventory management. In: Proceedings of the 12th International Conference on Precision Agriculture, Sacramento, CA, USA; 2014. p. 20–23.

15. Guldogan O, Rotola-Pukkila J, Balasundaram U, Le TH, Mannar K, Chrisna TM, et al. Automated tree detection and density calculation using unmanned aerial vehicles. In: 2016 Visual Communications and Image Processing (VCIP). IEEE; 2016. p. 1–4.

16. Gnädinger F, Schmidhalter U. Digital counts of maize plants by unmanned aerial vehicles (UAVs). Remote sensing. 2017;9(6):544. doi: 10.3390/rs9060544

17. Weinacker H, Koch B, Weinacker R. TREESVIS: A software system for simultaneous ED-real-time visualisation of DTM, DSM, laser raw data, multispectral data, simple tree and building models. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. 2004;36:90–95.

18. Kattenborn T, Sperlich M, Bataua K, Koch B. Automatic single tree detection in plantations using UAV-based photogrammetric point clouds. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences. 2014;40(3):139. doi: 10.5194/isprsarchives-XL-3-139-2014

19. Wallace L, Lucieer A, Watson CS. Evaluating tree detection and segmentation routines on very high resolution UAV LiDAR data. IEEE Transactions on Geoscience and Remote Sensing. 2014;52(12):7619–7628. doi: 10.1109/TGRS.2014.2315649

20. Mohan M, Silva C, Klauberg C, Jat P, Catts G, Cardil A, et al. Individual tree detection from unmanned aerial vehicle (UAV) derived canopy height model in an open canopy mixed conifer forest. Forests. 2017;8(9):340. doi: 10.3390/f8090340

21. Srestasathiern P, Rakwatin P. Oil palm tree detection with high resolution multi-spectral satellite imagery. Remote Sensing. 2014;6(10):9749–9774. doi: 10.3390/rs6109749

22. Malek S, Bazi Y, Alajlan N, AlHichri H, Melgani F. Efficient framework for palm tree detection in UAV images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2014;7(12):4692–4703. doi: 10.1109/JSTARS.2014.2331425

23. Lowe DG. Distinctive image features from scale-invariant keypoints. International journal of computer vision. 2004;60(2):91–110. doi: 10.1023/B:VISI.0000029664.99615.94

24. Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006;70(1-3):489–501. doi: 10.1016/j.neucom.2005.12.126

25. LeCun Y, Bengio Y, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks. 1995;3361(10):1995.

26. LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015;521(7553):436. doi: 10.1038/nature14539 26017442

27. Pan SJ, Yang Q, et al. A survey on transfer learning. IEEE Transactions on knowledge and data engineering. 2010;22(10):1345–1359. doi: 10.1109/TKDE.2009.191

28. Reyes AK, Caicedo JC, Camargo JE. Fine-tuning Deep Convolutional Networks for Plant Recognition. In: CLEF (Working Notes); 2015.

29. Amara J, Bouaziz B, Algergawy A, et al. A Deep Learning-based Approach for Banana Leaf Diseases Classification. In: BTW (Workshops); 2017. p. 79–88.

30. Mohanty SP, Hughes DP, Salathé M. Using deep learning for image-based plant disease detection. Frontiers in plant science. 2016;7:1419. doi: 10.3389/fpls.2016.01419 27713752

31. Huang H, Deng J, Lan Y, Yang A, Deng X, Zhang L. A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery. PloS one. 2018;13(4):e0196302. doi: 10.1371/journal.pone.0196302 29698500

32. Kussul N, Lavreniuk M, Skakun S, Shelestov A. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters. 2017;14(5):778–782. doi: 10.1109/LGRS.2017.2681128

33. Mortensen AK, Dyrmann M, Karstoft H, Jørgensen RN, Gislum R, et al. Semantic segmentation of mixed crops using deep convolutional neural network. In: CIGR-AgEng Conference, 26-29 June 2016, Aarhus, Denmark. Abstracts and Full papers. Organising Committee, CIGR 2016; 2016. p. 1–6.

34. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems; 2012. p. 1097–1105.

35. Li W, Fu H, Yu L, Cracknell A. Deep learning based oil palm tree detection and counting for high-resolution remote sensing images. Remote Sensing. 2016;9(1):22. doi: 10.3390/rs9010022

36. Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture. 2018;147:70–90. doi: 10.1016/j.compag.2018.02.016

37. Rebetez J, Satizábal H, Mota M, Noll D, Büchi L, Wendling M, et al. Augmenting a convolutional neural network with local histograms—a case study in crop classification from high-resolution UAV imagery. In: European Symp. on Artificial Neural Networks, Computational Intelligence and Machine Learning; 2016. p. 515–520.

38. Jensen JR, Lulla K. Introductory digital image processing: a remote sensing perspective. 1987.

39. Yang CC. Image enhancement by modified contrast-stretching manipulation. Optics & Laser Technology. 2006;38(3):196–201. doi: 10.1016/j.optlastec.2004.11.009

40. Daily M. Hue-saturation-intensity split-spectrum processing of Seasat radar imagery. Photogrammetric Engineering and Remote Sensing. 1983;49:349–355.

41. Bannari A, Morin D, Bonn F, Huete A. A review of vegetation indices. Remote sensing reviews. 1995;13(1-2):95–120. doi: 10.1080/02757259509532298

42. Xue J, Su B. Significant remote sensing vegetation indices: A review of developments and applications. Journal of Sensors. 2017;2017. doi: 10.1155/2017/1353691

43. Ren S, He K, Girshick R, Sun J. Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems; 2015. p. 91–99.

44. Sa I, Ge Z, Dayoub F, Upcroft B, Perez T, McCool C. Deepfruits: A fruit detection system using deep neural networks. Sensors. 2016;16(8):1222. doi: 10.3390/s16081222

45. Jin S, Su Y, Gao S, Wu F, Hu T, Liu J, et al. Deep learning: individual maize segmentation from terrestrial lidar data using faster R-CNN and regional growth algorithms. Frontiers in plant science. 2018;9:866. doi: 10.3389/fpls.2018.00866 29988466

46. Bargoti S, Underwood J. Deep fruit detection in orchards. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2017. p. 3626–3633.

47. Pix4D S. Pix4dmapper; 2014.

48. Turner D, Lucieer A, Watson C. An automated technique for generating georectified mosaics from ultra-high resolution unmanned aerial vehicle (UAV) imagery, based on structure from motion (SfM) point clouds. Remote sensing. 2012;4(5):1392–1410. doi: 10.3390/rs4051392

49. Schowengerdt RA. Remote sensing: models and methods for image processing. Elsevier; 2006.

50. Hunt ER, Daughtry C, Eitel JU, Long DS. Remote sensing leaf chlorophyll content using a visible band index. Agronomy Journal. 2011;103(4):1090–1099. doi: 10.2134/agronj2010.0395

51. Mckinnon T, Hoff P. Comparing RGB-based vegetation indices with NDVI for drone based agricultural sensing. Agribotix Com. 2017; p. 1–8.

52. Vergeiner C, Banala S, Kräutler B. Chlorophyll breakdown in senescent banana leaves: catabolism reprogrammed for biosynthesis of persistent blue fluorescent tetrapyrroles. Chemistry–A European Journal. 2013;19(37):12294–12305. doi: 10.1002/chem.201301907

53. Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer; 2014. p. 818–833.

54. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. 2014.

55. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 2818–2826.

56. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2014. p. 580–587.

57. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, et al. Tensorflow: A system for large-scale machine learning. In: 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16); 2016. p. 265–283.

Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV)

Souhrn

Klíčová slova:

Zdroje

PLOS One

Aktuální možnosti diagnostiky a léčby litiáz