Detecting spikes of wheat plants using neural networks with Laws texture energy
© The Author(s) 2017
Received: 16 March 2017
Accepted: 2 October 2017
Published: 13 October 2017
The spike of a cereal plant is the grain-bearing organ whose physical characteristics are proxy measures of grain yield. The ability to detect and characterise spikes from 2D images of cereal plants, such as wheat, therefore provides vital information on tiller number and yield potential.
We have developed a novel spike detection method for wheat plants involving, firstly, an improved colour index method for plant segmentation and, secondly, a neural network-based method using Laws texture energy for spike detection. The spike detection step was further improved by removing noise using an area and height threshold. The evaluation results showed an accuracy of over 80% in identification of spikes. In the proposed method we also measure the area of individual spikes as well as all spikes of individual plants under different experimental conditions. The correlation between the final average grain yield and spike area is also discussed in this paper.
Our highly accurate yield trait phenotyping method for spike number counting and spike area estimation, is useful and reliable not only for grain yield estimation but also for detecting and quantifying subtle phenotypic variations arising from genetic or environmental differences.
Wheat is one of the three most important crop species worldwide with 700 million tonnes of grain produced annually . However, with population growth, increasing demand and climate change threatening supply, greater effort is needed to ensure sustainable wheat crop production . This translates into increased pressure on plant breeders to rapidly and accurately identify suitable wheat plant varieties that could be used for commercial production. In this effort, crop phenotyping by quantitative assessment of crop canopy features plays an important role as a quantifier of crop performance. It thus represents an important tool for identifying high-yielding novel varieties. One of the aims of digital crop phenotyping is to predict, non-destructively, the yield of a crop and preferably at an early stage in plant development.
The life span of cereal plants can be divided into four stages [3, 4] based on the Feekes scale: tillering, stem elongation, heading and ripening. Of the critical factors contributing to crop yield, tiller number is established at the early stage while spike number features in the mid-life of plant development. Other factors such as spike size, grain number per spike and grain weight feature at later stages. One aim of the phenotyping process is to understand plant development over time and its relevance to final yield. If, however, yield can be estimated at an early stage using early indicators alone, then the length of experiments can be reduced which would potentially accelerate breeding efforts; it would certainly reduce the cost per trial. However, to achieve this requires complex growth models that link early or middle plant development to final yield. Tillers are important initial components related to yield as they have the potential to develop grain-bearing spikes. However, the number of tillers a plant develops is not constant and will vary due to the interaction between genetic makeup, environmental conditions and agricultural practice. In this study, we focus on the heading growth stage and one of the yield measures—spike number, rather than tiller number. This will serve as the basis for a top-down approach to plant and growth modelling to be implemented later.
To date, there have been relatively few studies concerned with spike detection and specific characterisation [5–8]. Some spike characteristics, such as awn number, awn length and spike length were measured in wheat using morphological image processing of images taken of single spikes in order to classify the wheat variety in question [5, 6]. Lv  developed a spike identification method based on a back propagation neural network using Hu moments that measured seven characteristic parameters with images of individually cut spikes. A similar destructive spike measurement method was proposed by Hongju and Changing . However, these methods are unsuitable for high-throughput, non-destructive, phenotyping for the purpose of identifying spikes from whole living wheat plants.
In this paper, we propose a novel approach for detecting spikes from digital images of wheat plants. We have observed that there is a difference in the texture features between spikes and leaves despite their colors being similar. This is particularly true at the early heading stage, where texture is defined as the spatial arrangement of color or intensity in a region of interest. Therefore, we propose to use Laws texture energies as texture features and use neural network for spike detection. The major advantage of our approach is that it is non-destructive and a high-throughput approach for spike detection which opens the door for phenotyping of spike traits in time sequences of plant images.
Results and discussion
Spike identification on living plants (single time point)
Results of counting the number of spikes
Number of spikesa
No of imagesb
Evaluation of spike detection
Total number of imagesa
We remark here that the evaluation thus far was based on individual images of plants at a single time point. However, using a time-series of images as well as images of the same plant from different perspective could improve the accuracy further.
Measuring the growth of individual spikes
Measuring the growth of spike area of whole plants (Mace in small pot)
Measuring the growth of spikes under different nitrogen treatments
Prediction of yield based on spike size
One of the principle aims of image-based phenotyping is to quantify plant traits non-destructively as a function of plant genotype, environmental conditions and time. However, it is also theoretically possible for image-based information derived at early stages of plant development to be utilised for the prediction of plant traits at later stages. In this study, we demonstrate this possibility by predicting final grain yield based on the spike size data obtained at the early heading stage.
There is, however, one major issue that needs to first be resolved for the purpose of making an absolute yield prediction. The issue relates to the spike number of a plant at the last imaging day being potentially different from the final number at day of harvest. To deal with this issue, we may focus only on predicting the average grain yield per spike, instead of the final total yield, using the size of the first spike of a plant for that analysis.
The power law model allowed us to predict grain yield per spike essentially at an early stage. It is also possible to estimate the grain yield potential of a whole plant using the combination of spike numbers and spike size. In the cases that were studied, we were actually able to predict yield using data as at the 8th day after the first spike became visible. We also find that the accuracy of the prediction can be further improved slightly if we measure for a longer period (i.e., up to 17 days after spike emergence).
We present an effective method for cereal spike detection from digital images. We employ the neural network based method with Laws texture energy for spike detection . The proposed approach has been evaluated on plant images achieving an accuracy higher than 80% in the identification of wheat spikes. We also demonstrated that the proposed method is able to determine both the number of spikes and the spike growth, which we indicate can be useful to quantify phenotypic traits for genetic variation and for treatment effects. Spike detection and final grain yield per spike were found to be highly correlated, providing the user with a potential algorithm with which to estimate final grain yield at earlier developmental stages. From an application prospective, the methodology can conceivably be used to estimate grain yield in the field. Consequently, it is possible to predict final grain yield at least 50–60 days prior to harvesting, which could provide growers with early decision making opportunities for additional practices. For example, if predicted yield is higher than originally designed, later unnecessary application of nitrogen fertilizer may be avoided (unless aimed to boost grain protein content). Finally, the proposed approach has the potential to be applied to other cereal crops such as barley and rice, and the concepts can also form the basis of a solid platform for non-cereal crops.
In future work, we shall explore the possibility of taking greater advantage of time-course image sequences of plants grown in individual pots to further improve the performance of spike detection method. A natural but ambitious extension we shall also consider is to adapt the algorithm to suit spike detection in field situations, which is arguably a more relevant enterprise addressing the need of plant breeders and cereal crop agriculture generally.
Plant material and growth condition
The 2013 dataset
Australian spring wheat cultivars Gladius and Kukri were grown in pots in glasshouse conditions between January and June, 2013. Preselected seeds of similar size were sown in pots filled with 2.5 kg of soil mix (coco-peat based potting media containing with different amount of N). Nitrogen was applied at sowing as urea at 10 mg (n1), 25 mg (n2), 75 mg (n3), 150 mg (n4), and 450 mg (n5) N/kg of soil. Four week old plants were phenotyped using the LemnaTec Scanalyzer 3D imaging system. RGB images were automatically captured daily for another 30 days. The plants were grown to maturity and harvested for their biomass and grain yield.
The 2014 dataset
Two common Australian bread wheat cultivars (Mace, Emu Rock) were grown in a coco-peat based potting mix containing slow release fertiliser (Osmocote). Multiple seeds were planted in 2.5 and 4.5 L draining pots and thinned out at the two-leaf stage to a single plant or four plants per pot, respectively. Seeds were planted on 20-12-2013 and grown and watered to bench capacity until 20-1-2014. Plants were then loaded onto The Plant Accelerator’s LemnaTec imaging system (LemnaTec GbmH, Aachen, Germany) for automated imaging and watering until 5-3-2014. Watering was maintained at 35% (w/w) gravimetric water content for the duration of the experiment. Average greenhouse temperatures were 25 °C during the day and 20 °C at night.
Visible light RGB images of wheat were taken daily using a LemnaTec 3D Scanalyzer (LemnaTec, GmbH, Aachen, Germany) at the Plant Accelerator® (TPA), the University of Adelaide. At each time point three images were taken, two side view images at 90° horizontal rotation and a top view image. RGB images were taken with a Basler Pilot GigE Vision Camera (piA2400-12 gm/gc) with a 2454 × 2056 resolution and stored in PNG format. Since the side view images provided more information compared to the top view images, we only used side view images in this study.
Image processing pipeline for spike detection
Two of the important objectives of this study were to detect the emergence of spikes and to count the number of spikes present. To avoid the problem of single spikes appearing as separate regions, the morphological closing (the dilation and then erosion) algorithm was used to integrate regions belonging to the same head into one region. To obtain the correct number of detected spikes it is necessary to resolve any issue arising from spike overlap (with leaves or other spikes). As individual spikes generally have similar size and shape, as shown in Fig. 6, we used the average size as a criterion to detect spike overlap. Two geometric parameters, the average area and perimeter, were used for this task. If the size of a detected region is deemed too large, relative to the average spike size, it is classified as overlapped spikes and treated as two spikes. In this way, we can reduce spike counting error from the initial counting. An example of the final result is shown in Fig. 6d.
Image segmentation using colour indices
Spike detection with neural network based Laws texture energy method
Texture analysis is an important specific methodology in computer image analysis for classification, detection and segmentation. Some of the most popular texture feature extraction methods are based on grey level co-occurrence statistics [15, 16], wavelet packets approaches , filtering methods like morphological filters, Fourier filters, random field models , Gabor filters  and local binary patterns . Each method offers certain advantages and some disadvantages in discriminating texture characteristics. The method of choice depends on the problem at hand.
Accuracy of the classification
Spike samples (pixels)
Leaf samples (pixels)
TP ratea (%)
TN ratea (%)
Spike detection result refinement
As shown in Fig. 8, for dataset 2013, as spikes grow above the frame in the image, we simply extracted the region above the blue frame in order to remove the noise (Fig. 8e). But for dataset 2014, blue frames were not used in the experiment. In order to remove most noise on the bottom, we defined a circle centred at the centre of the top line of the pot, and the radius was calculated as 60% of the total height of the plant (Fig. 8f). Pixels within the circle will be classified as non-spike pixels.
QL wrote the software and algorithms for processing the images; JC, QL and SM were involved in the development of algorithms. BB and MO designed and performed the experimental work, and acquired images. JC and SM drafted the manuscript and all authors contributed equally to the editing and improving the final manuscript.
The authors would like to thank the Plant Accelerator for providing the image data for this research and Sebastian Kipp for assistance with the 2014 project. The Plant Accelerator, Australian Plant Phenomics Facility, was funded under the National Collaborative Research Infrastructure Strategy. QL would also like to acknowledge the China Scholarship Council (CSC) for funding her visit to the PBRC, University of South Australia, during which time this research was conducted.
The authors declare that they have no competing interests.
Availability of data and materials
The Matlab programs and sample data are available from https://sourceforge.net/projects/spike-detection.
Consent for publication
Consent and approval for publication was obtained from all authors. Consent for the use of plant images and manual measured yields was obtained from University of South Australia and Adelaide University.
Ethics approval and consent to participate
This project was supported in part by the Australian Research Council under Grant LP150100055.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- FAOSTAT. http://faostat3.fao.org/faostat-gateway/go/to/browse/Q/QC/E.
- Tester M, Langridge P. Breeding technologies to increase crop production in a changing world. Science. 2010;327:818–22.View ArticlePubMedGoogle Scholar
- Herbek J, Lee C. A comprehensive guide to wheat management in Kentucky. http://www.uky.edu/Ag/GrainCrops/ID125Wheat_Management_Kentucky.html.
- Kirby EJM. Analysis of leaf, stem and ear growth in wheat from terminal spikelet stage to anthesis. Field Crops Res. 1988;18:127–40.View ArticleGoogle Scholar
- Bi K, Jiang P, Li L, Shi B, Wang C. Non-destructive measurement of wheat spike characteristics based on morphological image processing. Trans Chin Soc Agric Eng. 2010;12:212–6.Google Scholar
- Bi K, Jiang P, Wei T, Huang F, Wang C. The design of wheat variety BP classifier based on wheat ear feature. Chin Agric Sci Bull. 2011;6:464–8.Google Scholar
- Lv X. Wheat spike recognition based on image processing technology, Master Thesis, Hebei Agricultural University, 2012.Google Scholar
- Hong H, Ji C. Based on image processing technology of Ear of Wheat’s production methods of measurement. J Agric Mach. 2007;38(12):116–9.Google Scholar
- Foulkes MJ, Slafer GA, Davies WJ, Berry PM, Sylvester-Bradley R, Martre P, Calderini DF, Giffiths S, Reynolds MP. Raising yield potential of wheat. III. Optimizing partitioning to grain while maintaining lodging resistance. J Exp Botany. 2011;62:469–86.View ArticleGoogle Scholar
- Wendroth O, Reuter HI, Kersebaum KC. Predicting yield of barley across a landscape: a state-space modeling approach. J Hydrol. 2003;272(1–4):250–63.View ArticleGoogle Scholar
- Meyer GE, Neto JC. Verification of color vegetation indices for automated crop imaging applications. Comput Electron Agric. 2008;63(2):282–93.View ArticleGoogle Scholar
- Woebbecke DM, Meyer GE, Mortensen DA. Color indices for weed identification under various soil, residue and lighting conditions. Trans ASAE. 1995;38:259–69.View ArticleGoogle Scholar
- El-Faki MS, Zhang N, Peterson DE. Factors affecting color-based weed detection. Trans ASAE. 2000;43:1001–9.View ArticleGoogle Scholar
- Yang CC, Prasher SO, Landry J, Ramaswamy HS. Development of an image processing system and a fuzzy algorithm for site-specific herbicide applications. Precis Agric. 2003;4:5–18.View ArticleGoogle Scholar
- Gotllieb EE, Kreysig HE. Texture descriptors based on Co-Occurrence matrices. Comput Vis Graph Image Proc. 1990;51:70–86.View ArticleGoogle Scholar
- Haralick RM. Statistical and structural approaches to texture. Proc IEEE. 1979;67:786–804.View ArticleGoogle Scholar
- Chang T, Kuo C. Texture analysis and classification with tree-structure wavelet transform. IEEE Trans Image Proc. 1993;2(4):429–41.View ArticleGoogle Scholar
- Chellappa R, Chatterjee S. Classification of textures using gaussian markov random fields. IEEE Trans Acoust Speech Signal Process. 1985;33(4):959–63.View ArticleGoogle Scholar
- Fogel I, Sagi D. Gabor filters as texture discriminator. Biol Cybern. 1989;61(2):103–13.View ArticleGoogle Scholar
- Ojala T, et al. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell. 2002;24(7):971–87.View ArticleGoogle Scholar
- Laws KI. Textured image segmentation. PhD. thesis. University of Southern California, 1980.Google Scholar
- Ching-Fen J, Chen ML. Segmentation of ultrasonic ovarian images by texture features. In: The 20th annual international conference of the IEEE engineering in medicine and biology society, 1998; 20(2): 850–853.Google Scholar
- Mougiakakou S, Golemati S, Gousias I, Nicolaides A, Nikita K. Computer-aided diagnosis of carotid atherosclerosis based on ultrasound image statistics, laws’ texture and neural networks. Ultrasound Med Biol. 2007;33(1):26–36.View ArticlePubMedGoogle Scholar
- Christodoulou CI, Pattichis CS, Pantziaris M, Nicolaides A. Texture-based classification of atherosclerotic carotid plaques. IEEE Trans Med Imaging. 2003;22(7):902–12.View ArticlePubMedGoogle Scholar