Estimating rice chlorophyll content and leaf nitrogen concentration with a digital still color camera under natural light

Background The color of crop leaves is closely correlated with nitrogen (N) status and can be quantified easily with a digital still color camera and image processing software. The establishment of the relationship between image color indices and N status under natural light is important for crop monitoring and N diagnosis in the field. In our study, a digital still color camera was used to take pictures of the canopies of 6 rice (Oryza sativa L.) cultivars with N treatments ranging from 0 to 315 kg N ha-1 in the field under sunny and overcast conditions in 2010 and 2011, respectively. Results Significant correlations were observed between SPAD readings, leaf N concentration (LNC) and 13 image color indices calculated from digital camera images using three color models: RGB, widely used additive color model; HSV, a cylindrical-coordinate similar to the human perception of colors; and the L*a*b* system of the International Commission on Illumination. Among these color indices, the index b*, which represents the visual perception of yellow-blue chroma, has the closest linear relationship with SPAD reading and LNC. However, the relationships between LNC and color indices were affected by the developmental phase. Linear regression models were used to predict LNC and SPAD from color indices and phasic development. After that, the models were validated with independent data. Generally, acceptable performance and prediction were found between the color index b*, SPAD reading and LNC with different cultivars and sampling dates under different natural light conditions. Conclusions Our study showed that digital color image analysis could be a simple method of assessing rice N status under natural light conditions for different cultivars and different developmental stages. Electronic supplementary material The online version of this article (doi:10.1186/1746-4811-10-36) contains supplementary material, which is available to authorized users.


Background
Nitrogen (N) is one of the most important nutrients essential for the growth of crops, and is a major component of chlorophyll and protein which are closely associated with leaf color, crop growth status and yield [1]. Insufficient N supply leads to smaller leaves, lower chlorophyll content and less biomass production, and consequently, reduced grain yield and quality [2,3]. Conversely, excessive N application can lead to the environmental problems of water and atmospheric pollution [4]. Hence, measuring crop N status timely is critical for increasing N use efficiency and environmental quality [5,6].
The spectral reflectance of crop leaf or canopy is known to be correlated with N status [7,8]. The instruments for measuring spectral reflectance are the chlorophyll meter [9][10][11], multi-spectral sensor [12], hyper-spectral sensor [7,8] and commercial digital camera [13][14][15], are used in precision agriculture for growth monitoring, nitrogen diagnosis and site-specific crop management. The SPAD meter (SPAD-502, Minolta Camera Co., Osaka, Japan), measures leaf chlorophyll content nondestructively, has potential for improving N use efficiency without affecting grain yield in real-time nitrogen management (RTNM) experiments [9,11]. A disadvantage of the SPAD-502 for assessing crop N status is its small sampling area (6 mm 2 ). In addition, the measurements are subject to operator bias so a large number of repetitions are needed to obtain reliable results [16,17], and SPAD meter experiences difficulties in distinguishing chlorophyll levels when crops are near or above the optimum N supply [10,18]. In contrast, satellite or airborne-mounted hyperspectral sensors can obtain spectral information in a larger sampling area and record more spectral bands [7,8,12]. The high cost of images, infrequent satellite overpasses and risk of images being obscured by clouds limit the application of these platforms for commercial use [19]. The proximal sensors GreenSeeker® (NTech Industries Inc., USA) and Yara N-sensor (Yara International ASA, Germany) which measure red and near infra-red (NIR) reflectance, overcome some of the limitations of satellite or airborne sensors [20], but their accuracy is influenced by background soil interference [8].
Alternatively, images from digital still color cameras, which record spectral information of visible bands, have a low cost but very high image resolution (consumer cameras in 2014 record up to 40 megapixels per image). At a sensor height of about 1 meter above the canopy, high-spatial-resolution images separate crops from background soil or other interferences, which is important for accurate diagnosis of N status when the vegetation fraction was low [13,21]. Moreover, images from digital still cameras contain a large amount of information about the crop structure and leaf color, such as leaf orientation, plant height, biomass accumulation and leaf senescence [22][23][24], and these parameters are easy to obtain [13,25] with existing software, such as MatLab® (MathWorks Inc.), the free-ware package ImageJ [26]. Previous studies showed that canopy cover estimated from the images was not only highly correlated with leaf area index (LAI), aboveground biomass and N accumulation [13,27], but was also stable in varying environmental conditions [28,29]. In addition to canopy cover, color digital images provide spectral information in the visible bands which are closely related to the leaf N concentration (LNC) and SPAD readings [17]. Hunt et al. [30] found that the triangular greenness index (TGI), which was derived from red, green and blue bands of a digital still color camera, was sensitive to leaf chlorophyll content of a whole canopy.
The color-related indices from digital still cameras can diagnose crop N status [17,30,31]. Previous studies on the analysis of leaf color were mostly undertaken in controlled light conditions [17,[32][33][34]. This approach could reduce the impact of light on the image color, and easily obtain a reliable relationship between N status and leaf color indices. However, the results derived from the controlled light cannot be completely applied to natural light because of the variable light conditions [15,35,36]. Besides, there are still many uncertainties in the use of digital still cameras for N diagnosis under natural light conditions, and further validation is necessary to ensure the application in the field.
In this study, experiments with different N application rates were carried out in the field under sunny and overcast conditions (1) to analyze the relationship between chlorophyll content, LNC and canopy color related indices in different cultivars and stage of phasic development, (2) to establish the possible models for the diagnosis of crop N status using image color indices, and (3) to validate the applicability of the models under different natural light conditions.

Correlation of color related indices and crop nitrogen status
Correlations between two rice N parameters (leaf N concentration and SPAD reading) and 13 image-color related indices (Eq. 1-10) were calculated with individual and pooled cultivars in 2011. Similar results were obtained from the three rice cultivars, i.e., Liangyoupeijiu, Nanjing45 and Nanjing46. Therefore, only correlation coefficients from the Liangyoupeijiu dataset (n = 72) and the pooled dataset (n = 240) were displayed in Table 1. SPAD readings showed significant correlations with each color index except g in both datasets. Among these color indices, L * , b * , R, G, B, r and INT were negatively correlated with SPAD readings, while the other indices were positively correlated with SPAD readings. Most of the color indices were closely correlated with LNC, however, the magnitude and direction of the correlation coefficients were not consistent with those between color indices and SPAD readings. Overall, the indices derived from the CIE L * a * b * color model had relatively higher correlation coefficients with SPAD readings and LNC. In this color model, the index b * represents the visual perception of yellow-blue chroma which is similar with the leaf color variation, and it has been used in many other studies for image color analysis [37,38]. Therefore, we select index b * as a representative for further analysis.
Relationships between SPAD, LNC and the color index b * Regression analyses were performed between SPAD readings, LNC and the color index b * using the 2011 dataset. Positive linear relationships were observed between LNC and SPAD readings, with the same trends in different sampling dates and cultivars ( Figure 1). The determination coefficient (R 2 ) in different sampling dates varied from 0.61 to 0.88 along with the root mean square error (RMSE) from 1.81 to 2.64 g kg −1 . The LNC decreased with the rice development, while the maximum SPAD values increased with rice growth. Smaller RMSE was obtained in jointing and booting stages than in vegetative and tillering stages. Similarly, Xue et al. [39] reported that the ratio index of NIR/green (R 810 /R 560 ) reached the best accuracy with LNC at jointing stage. When data were pooled across the sampling dates, there Table 1 Correlation coefficients between SPAD readings, leaf nitrogen concentration (LNC, g kg −1 ) and image color related indices (digital number from three color models: RGB, HSV and CIE L * a * b * )  was no significant trend observed between SPAD readings and LNC (Figure 1e). Significant negative linear relationships were seen between LNC and color index b * , with R 2 ranging from 0.58 to 0.86 in the four developmental stages ( Figure 2). The R 2 between b * and LNC at booting stage ( Figure 2d) was less than that between SPAD readings and LNC. Same as the relationship between SPAD readings and LNC, no significant trend was observed between b * and LNC when data pooled across the sampling dates ( Figure 2e). In addition, there were negative linear relationships between b * and SPAD readings ( Figure 3). The R 2 in the vegetative stage was lower than that in the other stages. There were obvious differences among cultivars for the relationship between b * and SPAD readings, especially the Liangyoupeijiu in jointing and booting stages (Figure 3c and d) and the Nanjing45 in vegetative stage (Figure 3a). In this case, the regression analysis was carried out with individual cultivars (Table 2). Overall, higher R 2 were observed from individual cultivars and sampling dates than that from the pooled dataset (Table 2). However, the intercepts and slopes of the linear relationship varied with rice cultivars, these differences might be partly caused by the different plant type among cultivars, with a loose shape and large mean leaf angle in hybrid indica rice (Liangyoupeijiu) [40,41] while tight shape and small mean leaf angle in japonica rice (Nanjing45, Nanjing46). The different plant type lead to different distribution of reflectance [42,43] and finally caused different imagederived indices.
In our experiments, image acquisition was carried out in the field under natural light near solar noon which was the period with the most stable illumination at the top of the atmosphere. This makes sure that the light intensity is not changing too much during image acquisition in a single day. The results from Table 2 indicated that reliable estimates of N status could be obtained from images taken under natural light. Considering the stability of N diagnosis in different locations and sampling dates, individual sampling dates could not meet the needs of crop monitoring and N diagnosis in various environmental conditions. Therefore, regression analysis was carried out with the pooled data of sampling dates. There were large differences of light intensity among different sampling dates even under overcast days (Table 3, PAR ranging from 145 to 692 μmol m −2 s −1 in 2011).
Interestingly, regression analysis did not show any evidence that the relationship between SPAD and color index b * was affected by the varying light intensity (Figure 3e). This might be attributed to the auto exposure controlled by the digital camera which adjusted the exposure time to make compensation for the amount of light reaching the image sensor. However, there were no significant trends between LNC and SPAD readings, or between LNC and color index b * using data pooled across different sampling dates (Figures 1e and 2e). Previous studies revealed that, for rice and corn, the relationship between LNC and SPAD readings could be improved simply by dividing the readings with specific leaf weight (SLW = dry leaf weight/leaf area) of the sampled leaves or introducing SLW as a second independent variable in the multiple regression [44][45][46]. The reason is that SPAD readings vary with leaf thickness which can be different in cultivars, developmental stages and environmental conditions [46,47], while the LNC has a relatively consistent value. The uncertain relationship between LNC and color index b * (Figure 2e) may also be caused by the difference of leaf thickness, because the color index b * and SPAD readings both reveal the spectral information of leaves, and their relationship keeps consistent with the pooled data of different sampling dates (Table 2 and Figure 3e). Nowadays, destructive sampling or hyperspectral-reflectance [48] is required for the measurement of SLW, however, including this defeats the purpose of using a cheap and simple digital camera.

Model calibration and validation
Since developmental stages in rice affected the response of N parameters to color indices, multiple linear regression analysis was carried out with the consideration of days after transplanting (DAT) to estimate LNC and SPAD ( Table 4). The multiple linear models highly improved the R 2 of SPAD readings and b * with LNC (Table 4, Figure 4a and b). The relationships between SPAD readings and b * were less affected by the process of development for all the cultivars (Table 2 and Figure 3e). Therefore, the consideration of DAT in the multiple linear regression did not improve the R 2 effectively (Figure 4c). The slope of the regression lines in Figure 4 was all less than 1, which indicated that predicted LNC or SPAD were generally smaller at the high value area and bigger at the low value area, than the observed ones. It was noteworthy that most of the predicted LNC in the vegetative stage (red symbols) were underestimated in Figure 4a and b. This is mainly because that the plants at this stage has higher concentration of N but lower concentration of chlorophyll. In this case, the lower concentration of chlorophyll would cause an underestimation of the LNC in the vegetative stage.
Validations were performed on the four models in Table 4 with all data obtained in 2010 under sunny days. The images used in model calibration and validation were taken under different weather conditions, the objective of this combination was to evaluate whether the model was robust under different light conditions. In general, good performances on the predicted models were observed for the estimation of LNC and SPAD ( Figure 5). Model 1 showed the best performance on the prediction of LNC with a normalized mean bias of 1.19% (Table 4 and Figure 5a). Model 2, 3 and 4 showed relatively lower R 2 and smaller negative bias. The model for the prediction of LNC with color index b * (Figure 5b) was not severely affected by the different light conditions in 2010 and 2011, compared to the prediction of LNC with SPAD ( Figure 5a). As with the calibration results, most of the data in the vegetative stage were below the 1:1 line in models 1 and 2. In addition, the data in tillering stage deviated from the 1:1 line in models 3 and 4, which resulted in the low R 2 ( Figure 5). The similar R 2  and RMSE in Figure 5c and d indicated that the relationship between SPAD and color index b * was not affected by the developmental stage. In our study, the light conditions during image acquisition were different between 2010 and 2011. Image acquisition in 2011 was under overcast days with low but stable diffused light. In contrast, images were taken under sunny days in 2010 with strong and variable (PAR ranging from 937 to 1759 μmol m −2 s −1 ) light (Table 3). We can remove the effect of illumination change on images when the change is over the whole image by normalizing the image [49] or using a calibration panel [36]. However, there were many other differences caused by the different light conditions. The strong illumination in 2010 caused many white spots and shadows which affected the image color (Additional file 1: Figure S1) while this phenomenon was hardly seen in the images taken in 2011. With the rice growth, the light status within rice canopy (transmittance, reflectance, absorption) becoming more and more complex, the change of illumination will cause different degrees of influence on canopies that in different height (Additional file 2: Figure S2). In this case, it is difficult to calibrate the image color accurately. Sakamoto et al. [15] calibrated image indices with the introduction of exposure value (EV) and obtained reliable camera retrieved vegetation indices (VIs). However, this method cannot apply to our study for the calibration of canopy color because of the different influence in one image.
Besides the light conditions, the prediction of crop N status with leaf color may be affected by many other environmental factors, such as developmental stage, diseases and drought stress [50]. These factors may be detectable from high-resolution canopy images [22,23]. Color indices analysis associated with these factors and other image characteristics (e.g. canopy cover, plant shape, leaf texture or even soil status), will provide more reliable results to N diagnosis. Further studies will be devoted to the exploration of image characteristics, leaf color correction and the calibration of the established model with physiological parameters (e.g. SLW) in the evaluation of N status under different environmental conditions.  Table 4 for the estimation of leaf nitrogen concentration (LNC, g kg −1 ) and SPAD. Model 1 (a), LNC = αSPAD + βDAT + γ, model 2 (b), LNC = αb * + βDAT + γ, model 3 (c), SPAD = αb * + βDAT + γ, model 4 (d), SPAD = αb * + γ. Different colors denote different developmental stages (red: vegetative, green: tillering, blue: jointing, purple: booting). Different symbols denote different cultivars used for model calibration (○ Liangyoupeijiu, Δ Nanjing45, □ Nanjing46).

Conclusions
Image color indices calculated from RGB, HSV and L * a * b * color models have significant correlations with SPAD readings and leaf N concentration (LNC) of rice leaves. Among these color indices, the index b * , which represents the visual perception of yellow-blue chroma, had the highest correlation coefficients with SPAD readings and LNC. Regression analysis showed significant linear relationships between index b * and N parameters. However, the relationship between LNC and SPAD reading, LNC and index b * were affected by the rice developmental stage. This is mainly caused by the leaf thickness which can be different in cultivars, developmental stages and environmental conditions. In this case, linear regression models were established between color index b * , LNC and SPAD readings by considering the developmental process in rice. The multiple linear models improved the R 2 of SPAD readings and b * with LNC, yet most of the predicted LNC in the vegetative stage were underestimated because of the inconsistent relationship between chlorophyll and N concentration. Validations on the models showed good performance and acceptable predicted precision with different cultivars and sampling dates under different natural light conditions. These results indicated that digital color image analysis could be a simple method for assessing rice N status under natural light conditions.

Experimental design
Two independent experiments with different N fertilization gradients were implemented in our study. Experiment I was a long-term site-specific rice-wheat rotation experiment that started in 1997. The trial comprised six fertilizer treatments represented as CK, N0, N1, N2, N3, and N4 for N application of 0, 0, 180, 225, 270 and 315 kg N ha −1 in rice season, respectively. Each treatment had four replicates which were arranged in a randomized block design. The data used in this paper were from the period May to November in 2010 and 2011 with cultivars Wuyunjing24 and Nanjing46, respectively. Experiment II was carried out in paddy fields with a rice-wheat rotation in 2010 and 2011. Six N application rates with three replicates were designed in this trial, which were represented as N0, N1, N2, N3, N4 and N5 with N application of 0, 120, 180, 240, 270 and 300 kg N ha −1 , respectively. The cultivars were Nanjing44 and Yangjing48 in 2010, and Nanjing45 and Liangyoupeijiu in 2011. For both experiments, the N was split into three applications, 40% as basal, 20% at tillering and 40% at booting. In addition, each plot received 90 kg K ha −1 and 20 kg P ha −1 except the CK treatment in experiment I. The applied K was split into 50% as basal and 50% at booting, and all the P was applied as basal fertilizer. Other crop managements were same as the local traditional practices.

Sample collection and digital image acquisition
For measuring rice growth and nutrition parameters, the above-ground part of rice plant was sampled about every two weeks after transplanting until the booting stage. A total of 4 sets of samples were collected in 2010 and 2011 ( Table 3). The plant samples were separated into leaves and stems (including sheaths), and dried at 105°C for half an hour and then at 70°C until constant weight. After that, the samples were weighed for dry weight and analyzed for leaf N concentration (LNC) by the Kjeldahl method [51]. Along with the plant sampling, a chlorophyll meter (SPAD-502, Minolta Camera Co., Osaka, Japan) was used to obtain SPAD values on the four youngest fully expanded leaves. Each blade was measured at three points: on the upper, middle and lower thirds on either side of the midrib. Then, average SPAD readings were calculated for each plot.
On the same day or following day of plant sampling, images of the rice canopy were captured using a digital still color camera (EOS 50D, Canon Inc.) with a resolution of 15 mega pixels. The camera was mounted on a tripod at the nadir position with a constant height of 1 m above the top of the rice canopy. Aperture priority mode was selected, and the camera was set at aperture of f/5.6, ISO of 100, white balance of 4,900 K, auto exposure and auto-focus with the flash turned off. In 2010, the pictures were taken at local time 12:00 -13:00 in sunny days, while in 2011, the pictures were taken at the same time period but on overcast days. In the days of picture taken (July and August), the deviation between local time and solar noon was within 4 minutes. All the pictures from the experiments were stored in CR2 (Canon raw image file) format. The photosynthetic active radiation (PAR) and illuminance were recorded by a portable light meter (GLZ-C, Top Instrument Co., Zhejiang, P. R. China) during the period of image acquisition. Average PAR was calculated with each set of pictures, and observed 937-1759 μmol photons m −2 s −1 and 145-692 μmol photons m −2 s −1 in 2010 and 2011, respectively (Table 3).

Image segmentation and color indices calculation
A raw image file contains minimally processed data from the image sensor of a digital camera. This file saves settings of white balance, color saturation, contrast and sharpness in it, but defers the processing. Therefore, all the modification made on a raw image file is nondestructive.
The canopy images in CR2 format were adjusted for white balance using the 18% gray card (R-27, Kodak) pictures which were taken simultaneously with the canopy images. Then, lens distortion correction was applied, and exposure was set to +1 for all images. After that, images were saved as joint photographic experts group (JPEG) files for further processing. All the procedures above were processed with Adobe Camera Raw (Adobe Systems Inc.).
Since the images contained the rice canopy and some non-canopy elements, such as soil, water and plant residues, images were segmented into canopy portion and non-canopy portion. A computer program was developed based on the G-R thresholding method [25,29] using MatLab® (MathWorks Inc.) to extract the canopy portion of the image. The G-R thresholding method was proposed according to the difference of reflectance spectrum between green vegetation and non-canopy elements in the visible band. There is a reflection peak for green vegetation in the green band, whereas no apparent change for soil or water albedo in the whole visible band. Therefore, the value of green channel minus that of red channel expands the difference between canopy and non-canopy portion.
After the image segmentation, 13 color indices derived from 3 color models were calculated. RGB model is the most common color model for the representation of digital images. A color in the RGB model is described by indicating how much of each of the red, green, and blue is included. The color is expressed as an RGB triplet (R, G, B), with the representation for black of (0, 0, 0) and for the brightest representable white of (255, 255, 255) in an 8-bit image [52]. R, G and B are the mean values of the red, green and blue channels, and r, g and b are the normalized RGB values, respectively. Intensity (INT) is the average of R, G and B. VI Green is a widely used vegetation index [13]. These indices were calculated as follows [27,29]: In addition, the CIE L * a * b * and HSV color spaces were also tested in this study. The L * coordinate in CIE L * a * b * [53] closely matches human perception of lightness, a * and b * dimensions represent the visual perception of red-green and yellow-blue chroma, respectively. Both a * and b * are independent with image lightness (L * ), and take on both negative and positive values (+a * reds, − a * greens, + b * yellows, − b * blues). The three coordinates of L * a * b * are computed from the tristimulus values X, Y and Z as following equations [32,54]: where X n , Y n and Z n describe a specified white objectcolor stimulus.
The HSV color space is represented as a cylindricalcoordinate in which the angle around the central vertical axis corresponds to hue (H). The calculation of H was listed below [27,29]: