Remote, aerial phenotyping of maize traits with a mobile multi-sensor approach

Background Field-based high throughput phenotyping is a bottleneck for crop breeding research. We present a novel method for repeated remote phenotyping of maize genotypes using the Zeppelin NT aircraft as an experimental sensor platform. The system has the advantage of a low altitude and cruising speed compared to many drones or airplanes, thus enhancing image resolution while reducing blurring effects. Additionally there was no restriction in sensor weight. Using the platform, red, green and blue colour space (RGB), normalized difference vegetation index (NDVI) and thermal images were acquired throughout the growing season and compared with traits measured on the ground. Ground control points were used to co-register the images and to overlay them with a plot map. Results NDVI images were better suited than RGB images to segment plants from soil background leading to two separate traits: the canopy cover (CC) and its NDVI value (NDVIPlant). Remotely sensed CC correlated well with plant density, early vigour, leaf size, and radiation interception. NDVIPlant was less well related to ground truth data. However, it related well to the vigour rating, leaf area index (LAI) and leaf biomass around flowering and to very late senescence rating. Unexpectedly, NDVIPlant correlated negatively with chlorophyll meter measurements. This could be explained, at least partially, by methodical differences between the used devices and effects imposed by the population structure. Thermal images revealed information about the combination of radiation interception, early vigour, biomass, plant height and LAI. Based on repeatability values, we consider two row plots as best choice to balance between precision and available field space. However, for thermography, more than two rows improve the precision. Conclusions We made important steps towards automated processing of remotely sensed data, and demonstrated the value of several procedural steps, facilitating the application in plant genetics and breeding. Important developments are: the ability to monitor throughout the season, robust image segmentation and the identification of individual plots in images from different sensor types at different dates. Remaining bottlenecks are: sufficient ground resolution, particularly for thermal imaging, as well as a deeper understanding of the relatedness of remotely sensed data and basic crop characteristics. Electronic supplementary material The online version of this article (doi:10.1186/s13007-015-0048-8) contains supplementary material, which is available to authorized users.


Examples for the observed skewness of three genotypes during the season
The skewness of the NDVI images indicates additional parameters?
The applicability of skewness as an additional parameter, quantifying the distribution of the greenness parameter NDVI ( Figure A5) within the image or within the segmented area, was investigated as well. The skewness of NDVI ( Figure A6) showed a different seasonal pattern than NDVI. The skewness of the NDVI Plot was relatively constant with a small reduction at the beginning of flowering and a stronger increase at the end of the season. Average values were usually close to zero. For NDVI Plant different phases were significantly more pronounced due to significant differences from normal distribution (> 0) in the early growth phase (371 °Cd) and particularly during the phase of senescence (> 940 °Cd). Repeatability of the skewness of NDVI was generally higher in larger plots (h 2 = one < two < three < four row plots) but did not always reflect the plot size differences. It strongly increased by the segmentation procedure. The repeatability of the skewness was generally lower as the one for the CC and NDVI and plot size effects were more pronounced making it less suited to differentiate among genotypes.
The NDVI skewness parameter may indicate whether an observed lower NDVI Plant is caused by lower overall leaf greenness or by ongoing senescence reducing the green leaf area (example in additional file 2 section 6). However, mixed pixels at the border between plant and soil may influence this signal, linking the skeweness of NDVI Plant also to differences in canopy cover. Indeed, the skewness of NDVI was linearly related to CC ( Figure A7) where the relationship was stronger for NDVI Plant . It can be assumed that the skewness of NDVI Plant mostly reflects the onset of senescence, but the mixed pixel and the senescence effect cannot be clearly disentangled here. Thus, we conclude that a pixel size of 2.5 cm on the ground (see Table 1) as in this study, is not sufficient to utilize the skewness as additional parameter.
We also tested, whether the distribution parameter the skewness of the NDVI Plant would be valuable as additional parameter. The skewness of NDVI Plant was a valuable indicator for nitrogen nutrition status in pearl millet [1]. It may be used as indicator, whether a low NDVI Plant is caused by a general low leaf greenness or by senescence, which increases the patchiness of green, yellow and brown leaf parts [2]. In our study, the explanatory power of skewness was limited: The increase in skewness with advanced senescence was likely related to a combination of senescent plant tissue (increased patchiness) and an increased portion of pixels containing a mixture of soil and plant signal. Thus, a high sensor resolution is a precondition to derive pure canopy pixels to measure senescence "patchiness" based on the skewness of NDVI Plant . Because the skewness of NDVI Plant is linked to the amount of mixed soil and plant pixels it might serve as a parameter for the quality of the segmentation.
However, if the NDVI distribution (skewness) is of interest it turned out to be important to have three or more maize rows for a high repeatability. For the reliable detection of CC at least two or better three rows are needed.
In conclusion the 'skewness', a parameter quantifying the histogram asymmetry of NDVI signals within images, proved very useful for evaluation of segmentation quality and development of senescence.

Figure A5
The skewness value (s) is a distribution parameter. A skewness value higher than zero indicates a shift towards low average NDVI values caused by a higher number of soil or senescent plant pixels. In contrast, skewness values below zero reflect a shift towards green pixels indicating a well-covered plot or plants in a fully green stage.