Open Access

A novel machine-vision-based facility for the automatic evaluation of yield-related traits in rice

Contributed equally
Plant Methods20117:44

https://doi.org/10.1186/1746-4811-7-44

Received: 18 October 2011

Accepted: 12 December 2011

Published: 12 December 2011

Abstract

The evaluation of yield-related traits is an essential step in rice breeding, genetic research and functional genomics research. A new, automatic, and labor-free facility to automatically thresh rice panicles, evaluate rice yield traits, and subsequently pack filled spikelets is presented in this paper. Tests showed that the facility was capable of evaluating yield-related traits with a mean absolute percentage error of less than 5% and an efficiency of 1440 plants per continuous 24 h workday.

Keywords

RiceYield-related traitsFast trait evaluationPlant phenotypingMachine vision

Background

Rice is the staple food for a large number of countries and regions in the world, particularly in Asia [1]. Because the world's population is increasing, obtaining higher yields has been the primary breeding target of rice cultivation [2]. As a complex agronomic trait, rice yield is determined by the product of the grain weight, the number of grains per panicle and the number of panicles per plant. The number of total spikelets per panicle and the seed setting rate are two traits that multiplicatively determine the number of grains per panicle, and the grain weight is largely determined by the grain size, including the grain length, the grain width, and the grain thickness [3].

The evaluation of yield traits, including the number of total spikelets (including filled and unfilled spikelets), the number of grains (also known as the number of filled spikelets), the seed setting rate (the number of filled spikelets divided by the number of total spikelets), the 1000-grain weight, the grain length, and the grain width, is an essential step in rice breeding, genetic research and functional genomics research [46]. Currently, rice yield trait evaluation is mainly performed by experienced workers. When investigations of large numbers of plants are needed, the manual measurement process is very subjective, inefficient, tedious, and error-prone. Most importantly, manual measurements are greatly affected by worker fatigue, which is a major problem in conducting mass measurements and renders the evaluation results questionable. In addition to trait extraction and evaluation, data logging and seed management are two instrumental steps in rice research. Traditionally, the processing of data, seed packaging, and seed coding are preformed manually and are thus error-prone and unreliable. A mistake in data management and seed management would lead to incorrect decisions and treatment of the seeds and is thus intolerable in rice research. For this reason, at least three workers are normally needed to check and verify the data to avoid the mistakes.

Modern plant breeding technologies are able to produce hundreds to thousands of new varieties, creating the need for rapid evaluation of plant materials to provide pertinent information prior to entering the next cycle of selection [7]. The low efficiency of manual trait evaluation makes it unsuitable for meeting the increasing demand for higher evaluation speeds. Automated assessment and measurement of plant phenotypes is therefore indispensable [8]. Several efforts have been made to automate plant phenotyping, such as automated analysis of plant leaves [9], roots [10, 11], hypocotyls [12], tillers [13], and shoot biomasses [14] and whole adult plants [15, 16]. Plant phenotyping facilities have also been established in large research centers and universities in Australia [17, 18], Germany [19], the UK [20], and France [21]. However, to the best of our knowledge, very little information is available on rice yield trait evaluation. Previously, our group developed a method to identify and count filled/unfilled spikelets [22]. We integrated visible-light imaging and X-ray imaging to simultaneously calculate the total spikelet number and the filled spikelet number. Nevertheless, the accuracy and efficiency of the method was limited by the performance of the X-ray system. Moreover, because of the utilization of an X-ray system, the prototype was expensive and posed a radiation hazard; thus, it was not suitable for widespread use. Research on single-trait evaluation, mainly grain dimension measurement, has also been reported [23, 24]. However, these studies have the disadvantage of measuring only a single trait.

This work aimed to develop an integrated and labor-free engineering solution for automatic panicle threshing, rice yield-related trait evaluation (including the number of total spikelets (NTS), the number of filled spikelets (NFS), the 1000-grain weight (TGW), the grain length (GL), and the grain width (GW)), and seed packing. The task involved the development of an automated threshing machine for the threshing of spikelets and the separation of spikelets and other unwanted materials; the design of mechanisms for separating the filled spikelets from the unfilled spikelets; the design of a machine vision system for imaging rice spikelets; the development of real-time algorithms for trait evaluation; the construction of a data logging and management system for data tracking; and the design of control and communication procedures to supervise the whole system, including a user-friendly interface.

Results and Discussion

Development of the SEA facility

The facility consisted of three major elements: a threshing unit, an inspection unit, and a packing-weighing unit (Figure 1). The control center used software developed using LabVIEW 8.6 (National Instruments, USA) to control the whole system. The operating procedure includes the following steps: (1) the barcode of the rice plant being evaluated was obtained with a barcode scanner or by manual input, depending on the user's selection. (2) The threshing machine was started as panicles to be processed by the machine were detected. Spikelets were transferred to the inspection unit while impurities were collected at the impurity outlet. (3) The "total-spikelet-vision camera" collected images of the total spikelets (total-spikelet image) that came from the threshing unit (including filled and unfilled spikelets). A wind separator separated the filled spikelets from the unfilled spikelets. The 'filled-spikelet-vision camera' collected images of the filled spikelets (filled-spikelet image). The images were analyzed to determine yield traits. Unfilled spikelets were collected at the unfilled-spikelet outlet. (4) After inspection, lifting equipment raised the collected filled spikelets and delivered them to a packing machine for packaging. (5) A code-jetting machine printed the barcode of the recently examined rice plant on the packing bag. (6) An electronic balance weighed the filled spikelets and sent the data to the computer. (7) A collecting device collected the packed filled spikelets. An additional movie file shows the operation procedure in more detail [see Additional file 1].The developed prototype, dubbed the Seed-Evaluation Accelerator (SEA), is shown in Figure 2.
Figure 1

Scheme of the SEA facility. Panicles were threshed by the threshing unit, and spikelets were transferred to the inspection unit. A wind separator separated filled spikelets from unfilled spikelets. One camera acquired images of the total spikelets (including filled spikelets and unfilled spikelets) and one acquired images solely of the filled spikelets. The images were subsequently analyzed to obtain yield traits. After inspection, the filled spikelets were packed and weighed.

Figure 2

The developed prototype of the SEA facility. The facility mainly consisted of three units: a threshing unit for removing spikelets from the panicles, an inspection unit for assessing and measuring yield traits, and a packing-weighing unit for packing and weighing filled spikelets.

The control flowchart is shown in Figure 3. The times required for panicle feeding (T f ), threshing (T t ), inspection (T i ), and packing-weighing (T p ) were designed to be 30, 50, 60, and 40 seconds, respectively. T idle was the idle time between subsequent measurements and depended on the operator. The threshing unit and the inspection unit worked in parallel with the packing-weighing unit, and Tp was less than T i . Thus, the total processing time per rice plant was determined by Eq. 1:
Figure 3

Control flowchart of the instrument.

T = T i + T i d l e
(1)

Threshing unit

Panicles were fed into the threshing unit via a panicle inlet. The threshing of the spikelets was triggered by pulses received from a photoelectric sensor attached to the panicle inlet. The machine threshed the spikelets through roller-compaction processes. A sieve was used to separate the spikelets from branches and other unwanted materials. A tilted, vibrating plate under the sieve disaggregated the spikelets as they reached the end of the plate. After the spikelets were vibrated into the spikelet outlet, impurities were blown out through the impurity outlet by an air blower.

Inspection unit

Figure 4 shows the details of the inspection unit. The prototype used two line-scan cameras (Spyder 3 GIGE vision SG-11-02k80-00-R, Teledyne DALSA Company, Germany) to acquire 5000 × 2048 pixel grayscale images with a resolution of 0.23 mm/pixel. The cameras were controlled by the computer workstation (HP z600, Hewlett-Packard Development Company, USA) through an Ethernet card (NI PCIe 8235, National Instruments Corporation, USA) that digitized the images into 8-bit files. Two line-array LED light sources served as the illumination system.
Figure 4

Details of the inspection unit. The inspection unit consisted mainly of a three-stage conveyor, two line-scan cameras, two LED light sources, a wind separator and an unfilled spikelet outlet.

Spikelets coming from the threshing unit were transferred onto the first conveyor (420 mm wide), where the ellipsoidal spikelets lay flat. When the spikelets were transferred to the second conveyor, they were imaged by the so-called "total-spikelet-vision camera" as they passed through the field of view of the camera. Then, the filled spikelets and the unfilled spikelets were separated by a wind separator mounted between the second and third conveyors. Unfilled spikelets were blown out and collected at an unfilled-spikelet outlet, whereas filled spikelets fell onto the third conveyor and were imaged by the "filled-spikelet-vision camera". To spread out the spikelets, the second conveyor was designed with a higher speed than that of the first conveyor, and the speed of the third conveyor was higher than that of the second. This design produced a separation of individual spikelets and consequently facilitated image analysis. A black conveyor belt was chosen, as it generated good contrast between the belt and spikelets and thus was beneficial for image segmentation.

Packing-weighing unit

As shown in Figure 5, after inspection, the filled spikelets fell into a grain-collecting tank placed under the third conveyor. Next, lifting equipment raised the tank and poured the filled spikelets into a packing machine. After packing, a code-jetting machine printed the barcode of the plant being evaluated onto the packing bag. Subsequently, an electronic balance weighed the filled spikelets and sent the grain weight (W grain ) to the computer. The TGW was obtained using Eq. 2:
Figure 5

Details of the packing-weighing unit. The packing-weighing unit consisted of lifting equipment, a packing machine, a code-jetting machine, an electronic balance and a collecting device.

T G W = ( W g r a i n × 1000 ) N F S
(2)

Communication interface

The communication interface is illustrated in Figure 6. The resultant total-spikelet image and the filled-spikelet image, along with the yield trait data, were displayed on the interface. When the "Input barcode manually?" button was clicked, a dialog box was shown to allow users to input the barcode manually.
Figure 6

Software interface of the implemented prototype.

Automated data and seed management

At the beginning of the yield trait evaluation for each rice plant, the user chose to either scan the barcode of the plant using a barcode scanner or input the barcode manually. This barcode was transferred to the code-jetting machine, which then sprayed the barcode on the packing bag to facilitate seed management. Yield trait data of the rice plant were stored in a Microsoft Excel file along with the barcode of the plant for indexing and data management.

Performance evaluation of the hardware

Rice panicles from 214 harvested Huageng 295 rice plants were tested to evaluate the performance of the prototype. All of the spikelets, including spikelets at the impurity outlet, the unfilled-spikelet outlet and the filled-spikelet outlet (packing machine), were collected. The number of total spikelets and the number of filled spikelets at the three outlets were counted separately and recorded. For the NTS, NFS and TGW, each rice plant was evaluated three times by different personnel, and the average values were computed as reference data. Manual observations were defined and computed as given in Table 1.
Table 1

Definition and calculation of manual observations

Variable

Definition

Calculation

NTS io

Number of total spikelets at the impurity outlet

 

NTS uo

Number of total spikelets at the unfilled-spikelet outlet

 

NTS fo

Number of total spikelets at the filled-spikelet outlet (packing machine)

 

NFS io

Number of filled spikelets at the impurity outlet

 

NFS uo

Number of filled spikelets at the unfilled-spikelet outlet

 

NFS fo

Number of filled spikelets at the filled-spikelet outlet (packing machine)

 

NTS manual

Manually measured value of the number of total spikelets (NTS of a rice plant)

NTS manual = NTS io + NTS uo + NTS fo

NFS manual

Manually measured value of the number of filled spikelets (NFS of a rice plant)

NFS manual = NFS io + NFS uo + NFS fo

NTS manual.image

Manually measured value of the number of total spikelets that were imaged by the camera

NTS manual.image = NTS uo + NTS fo

NFS manual.image

Manually measured value of the number of filled spikelets that were imaged by the camera

NFS manual.image = NFS uo + NFS fo

The threshing machine worked well during the tests. The absolute threshing error for total/filled spikelets was calculated as the total/filled spikelet number at the impurity outlet. The percentage threshing error for total/filled spikelets was calculated as the absolute threshing error divided by the number of total/filled spikelets of the rice plant being evaluated. Figure 7 ill-ustrates the percentage error of the threshing unit for total spikelets and filled spikelets. As shown in Figure 7, the threshing error for total spikelets was higher than for filled spikelets, indicating that it was more difficult to thresh unfilled spikelets than filled spikelets. It can be observed from Figure 7 that some samples had threshing errors that were significantly higher than the average error. This was because the feeding speed had an important influence on the threshing of the spikelets. Two panicles entering the threshing machine at the same time would lead to fewer spikelets being threshed. These spikelets that remained on the panicles would be blown out with the panicle branches as impurities and, consequently, increase the threshing error. The mean absolute error and mean absolute percentage error of the threshing unit were 34 and 3.33%, respectively, for total spikelets and 19 and 2.27%, respectively, for filled spikelets.
Figure 7

Performance of the threshing unit. The blue line and red line represent the percentage threshing errors of the threshing unit for total spikelets and filled spikelets, respectively.

The multi-stage conveyor proved able to segregate most of the spikelets. As a result, most kernels in both the total-spikelet image and the filled-spikelet image were observed to be isolated kernels. The packing-weighing unit worked properly during the test, with average weighing differences of less than 0.01 g between automatic measurements and manual measurements.

Performance evaluation of the image analysis algorithms

To test the performance of the image analysis algorithms, manually threshed spikelets of 90 Huageng 295 rice plants were fed into the inspection unit and imaged by the cameras. Manually threshed spikelets were used to exclude measuring errors caused by threshing. Figure 8 shows the results of manual observation versus image analysis of manually threshed spikelets. In comparison, the image-analysis performance with automatically threshed spikelets coming from the threshing unit was also investigated (Figure 9). The mean absolute error and the mean absolute percentage error with manually threshed spikelets were 22 and 1.36%, respectively, for the NTS and 7 and 0.54%, respectively, for the NFS. In comparison, the mean absolute error and the mean absolute percentage error with automatically threshed spikelets were 27 and 2.81%, respectively, for the NTS and 15 and 1.77%, respectively, for the NFS. As expected, the measuring accuracy for the manually threshed spikelets was higher than for the automatically threshed spikelets. This was because the threshing machine removed the hulls of some spikelets, and the broken hulls would appear in the image as impurities. The external appearance of some broken hulls was similar to a spikelet, and consequently, they would be mistakenly treated as spikelets. In addition, some brown rice spikelets (spikelets without hulls) were much smaller than normal spikelets and thus would be mistakenly treated as impurities.
Figure 8

Performance of the image analysis algorithms with manually threshed spikelets. Scatter plots of manual measurements versus automatic measurements with manually threshed spikelets for the number of total spikelets and the number of filled spikelets are shown. Ninety Huageng 295 rice plants were used as samples in the evaluation experiment. Least squares linear regression produced the following results: (a) number of total spikelets: line of best fit: y = 1.01x-2.85, correlation coefficient r = 0.9995, (b) number of filled spikelets: line of best fit: y = 0.998x-1.65, correlation coefficient r = 0.9998.

Figure 9

Performance of the image analysis algorithms with automatically threshed spikelets. Scatter plots of manual measurements versus automatic measurements with automatically threshed spikelets using the threshing unit for the number of total spikelets and the number of filled spikelets are shown. In total, 214 Huageng 295 rice plants were used as samples in the evaluation experiment. Least squares linear regression produced the following results: (a) number of total spikelets: line of best fit: y = 0.99x+6.07, correlation coefficient r = 0.993, (b) number of filled spikelets: line of best fit: y = 0.99x+4.47, correlation coefficient r = 0.997.

As measuring the GL and the GW of a grain is troublesome to perform, the mean GL and GW of only 50 rice plants were measured manually. For each rice plant, 10 filled spikelets were randomly chosen. Five workers measured the grain length and the grain width of each spikelet using a Vernier caliper, and the average value was regarded as the grain length and the grain width of one spikelet. To eliminate measuring errors caused by sampling, the facility measured the same 10 spikelets of each rice plant that were used for the manual measurements. The GL and GW of each rice plant were computed as the mean grain length and grain width values of the selected 10 spikelets.

Large variances were noted for the GL and GW estimation among different personnel. The lack of a mathematical definition of GL and GW and worker fatigue under continuous measuring conditions were believed to be two primary reasons for the huge variance among workers. Manual measurements and automatic measurements for GL and GW are illustrated in Table 2. Generally, automatically measured GL values were slightly larger than the values from manual measurements. However, GW values that were measured automatically matched well with those measured manually. The larger GL value from automatic measurement than from manual measurements was because there were errors in the manual location of the maximum length (GL), and inaccurate location of the length invariably leads to underestimation of the GL. Unlike GL measurements, imprecise location of the width may result in underestimation or overestimation of the GW. As a result, the average manually measured GW values matched well with automatically measured values.
Table 2

Grain length and grain width measured manually versus automatically (sample size = 50)

No.

Grain length (mm)

Grain width (mm)

 

Automatic measurement

Manual measurement

Absolute error

Relative error

Automatic measurement

Manual measurement

Absolute error

Relative error

1

7.34

7.31

0.03

0.46%

3.43

3.37

0.06

1.82%

2

7.32

7.35

0.03

0.36%

3.31

3.41

0.10

2.85%

3

7.23

7.10

0.13

1.88%

3.33

3.21

0.12

3.85%

4

7.43

7.31

0.12

1.65%

3.43

3.27

0.16

4.91%

5

7.19

7.14

0.05

0.66%

3.19

3.23

0.04

1.25%

6

7.45

7.19

0.27

3.69%

3.29

3.23

0.06

1.91%

7

7.31

7.26

0.06

0.76%

3.19

3.24

0.05

1.54%

8

7.12

7.09

0.03

0.38%

3.22

3.32

0.10

2.94%

9

7.38

7.29

0.08

1.14%

3.32

3.29

0.03

0.84%

10

7.39

7.31

0.08

1.14%

3.30

3.33

0.03

0.81%

11

7.41

7.20

0.21

2.97%

3.24

3.28

0.04

1.23%

12

7.40

7.21

0.19

2.62%

3.29

3.22

0.07

2.15%

13

7.25

7.13

0.12

1.61%

3.29

3.33

0.04

1.11%

14

7.16

7.13

0.03

0.46%

3.29

3.23

0.07

2.07%

15

7.39

7.23

0.15

2.14%

3.26

3.32

0.06

1.86%

16

7.51

7.31

0.20

2.76%

3.28

3.33

0.05

1.39%

17

7.19

7.13

0.06

0.91%

3.26

3.28

0.02

0.52%

18

7.46

7.29

0.16

2.25%

3.37

3.40

0.03

0.79%

19

7.27

7.28

0.01

0.14%

3.30

3.30

0.00

0.02%

20

7.29

7.20

0.09

1.28%

3.32

3.36

0.04

1.33%

21

7.32

7.23

0.09

1.29%

3.24

3.33

0.09

2.57%

22

7.33

7.25

0.07

0.98%

3.29

3.36

0.07

2.15%

23

7.52

7.37

0.15

2.09%

3.24

3.39

0.15

4.48%

24

7.38

7.32

0.06

0.85%

3.28

3.37

0.08

2.48%

25

7.33

7.33

0.00

0.01%

3.32

3.43

0.11

3.07%

26

7.36

7.22

0.14

1.94%

3.29

3.31

0.02

0.65%

27

7.36

7.19

0.17

2.39%

3.33

3.29

0.04

1.09%

28

7.28

7.29

0.01

0.14%

3.31

3.29

0.02

0.67%

29

7.24

7.18

0.06

0.81%

3.27

3.29

0.02

0.60%

30

7.42

7.28

0.15

2.00%

3.48

3.41

0.07

2.05%

31

7.33

7.27

0.06

0.82%

3.24

3.33

0.09

2.71%

32

7.29

7.11

0.19

2.61%

3.36

3.27

0.10

2.94%

33

7.47

7.35

0.12

1.68%

3.31

3.31

0.00

0.03%

34

7.26

7.24

0.02

0.21%

3.41

3.41

0.00

0.00%

35

7.59

7.42

0.17

2.22%

3.42

3.39

0.03

0.92%

36

7.31

7.18

0.13

1.80%

3.26

3.21

0.05

1.41%

37

7.36

7.20

0.16

2.16%

3.35

3.33

0.02

0.47%

38

7.50

7.26

0.23

3.22%

3.34

3.30

0.05

1.42%

39

7.36

7.28

0.08

1.09%

3.36

3.35

0.01

0.28%

40

7.12

7.13

0.02

0.26%

3.14

3.17

0.03

0.80%

41

7.34

7.15

0.19

2.67%

3.32

3.31

0.01

0.24%

42

7.21

7.26

0.05

0.69%

3.24

3.31

0.08

2.29%

43

7.39

7.20

0.19

2.64%

3.36

3.38

0.02

0.46%

44

7.37

7.24

0.13

1.75%

3.35

3.37

0.02

0.48%

45

7.38

7.32

0.06

0.78%

3.32

3.38

0.06

1.74%

46

7.24

7.20

0.03

0.46%

3.37

3.35

0.02

0.59%

47

7.42

7.26

0.16

2.23%

3.30

3.34

0.05

1.41%

48

7.59

7.39

0.21

2.79%

3.35

3.36

0.01

0.21%

49

7.38

7.32

0.05

0.73%

3.37

3.39

0.02

0.63%

50

7.28

7.23

0.05

0.68%

3.30

3.30

0.00

0.10%

Mean value

7.34

7.24

0.11

1.47%

3.31

3.32

0.04

1.31%

Performance evaluation of the whole facility

Scatter plots of manual measurements versus automatic measurements of the whole facility for the number of total spikelets, the number of filled spikelets and the 1000-grain weight are shown in Figure 10. As weighing differences between automatic measurements and manual measurements were minor, the discrepancy in the TKW between manual and automatic measurements was chiefly caused by NFS measurement differences.
Figure 10

Performance of the entire facility. Scatter plots of manual measurements versus automatic measurements with the facility for (a) the number of total spikelets, (b) the number of filled spikelets, and (c) the 1000-grain weight are shown. In total, 214 Huageng 295 rice plants were used as samples in the evaluation experiment. Least squares linear regression produced the following results: (a) number of total spikelets: line of best fit: y = 0.96x+10.24, correlation coefficient r = 0.992, (b) number of filled spikelets: line of best fit: y = 0.96x+6.14, correlation coefficient r = 0.996, (c) 1000-grain weight: line of best fit: y = 0.94x+1.56, correlation coefficient r = 0.91.

Table 3 summarizes the mean absolute error (MAE, defined by Eq. 3) and the mean absolute percentage error (MAPE, defined by Eq. 4) of the whole facility for the evaluated yield traits. As shown in the table, the facility was capable of evaluating yield traits with mean absolute percentage errors of 4.02%, 3.33%, 1.47%, 1.31% and 1.08% for number of total spikelets, number of filled spikelets, grain length, grain width, and 1000-grain weight, respectively. The MAE and MAPE were computed using Eqns. (3) and (4)
Table 3

Mean absolute error (MAE) and mean absolute percentage error (MAPE) for the evaluated traits (asample size = 214, bsample size = 50)

Trait

Definition

MAE

MAPE

NTSa

Number of total spikelets of a rice plant

41

4.02%

NFSa

Number of filled spikelets of a rice plant

29

3.33%

GLb

Average grain length of a rice plant

0.11 mm

1.47%

GWb

Average grain width of a rice plant

0.04 mm

1.31%

TGWa

1000-grain weight of a rice plant

0.26 g

1.08%

MAE = 1 n i = 1 n x i . a - x i . m
(3)
MAPE = 1 n i = 1 n x i . a - x i . m x i . m
(4)

where n was the number of samples, x i.a was the i th automatically measured value, and x i.m was the i th manually measured value.

As observed from the results, the measuring error of the NTS was larger than that of the NFS. This was because broken hulls caused by threshing were also imaged by the "total-spikelet-vision camera". Additionally, larger errors can result from broken hulls, which are very similar in appearance to complete spikelets and, consequently, may be treated as spikelets by the vision system but are ignored in manual observation. These broken hulls were blown out into the impurity outlet by the wind separator and thus would not influence the NFS measurement. Anther reason for the larger measuring error of the NTS was that the threshing error of the total spikelets was larger than that of the filled spikelets.

The threshing error directly decreases the number of spikelets that pass through the inspection unit and, consequently, influences the measuring accuracy of the NTS and the NFS. The relation between the threshing error and the measuring error of the NTS and the NFS were investigated. Figure 11 shows the variation of the percentage measuring error of the facility for the NTS and the NFS as a percentage threshing error changes. As shown in Figure 11, the measuring error for both the NTS and the NFS presented an upward trend as the threshing error increased. Compared with the measuring error for the NFS, the measuring error for the NTS had a weaker relationship with the threshing error. This was because broken hulls had a considerable effect on the NTS measuring error.
Figure 11

Relationship between the percentage measuring error of the facility and the percentage threshing error. (a) number of total spikelets and (b) number of filled spikelets.

Measuring the efficiency of the whole facility

Assuming T idle was 0 s, the total processing time per rice plant was approximately 60 s (calculated as Eq. 1); i.e., the implemented prototype was able to perform yield trait evaluations for approximately 1440 rice plants per 24 h continuous workday. Generally, an experienced worker can evaluate approximately 20 plants per day (working 8 hours per day). As the prototype needs no human operation except for panicle feeding, it is feasible to run the facility continuously for 24 hours for mass measurements. From this point of view, the facility was capable of evaluating rice yield traits with an efficiency more than 70 times greater than that of manual operation at its maximum throughput.

The efficiency of the whole facility was determined by the inspection unit and the time elapsed, which was limited by the threshing time. More effective threshing methods will be developed in the future to improve the efficiency of the whole facility. The software was designed to allow the images to be processed concurrently as the cameras were acquiring images. Image processing requires less time than image acquisition. The shorter time needed for image processing also opened up the possibility of applying more sophisticated algorithms for more sophisticated solutions. For instance, statistical classifiers such as the distance classifier, discriminant analysis and artificial neural networks could be adapted in the future to discriminate broken hulls from spikelets.

Automation and integration of threshing, multi-trait measurement and seed packing

With the only manual operation being panicle feeding, the SEA facility automated the entire process of threshing, fast multi-trait measurement and seed packing. The yield traits were automatically observed and stored with a unique code after system inspection. Meanwhile, the seeds were automatically packed with the relevant code. The automation and integration of the entire process will substantially improve the yield trait evaluation process for rice researchers. With the data tracking ability, it was convenient for the user to manage and analyze data. Data tracking also allowed the user to combine yield traits with other traits such as the tiller number, the leaf area, the plant height, thus allowing an integrated understanding. Moreover, data tracking is beneficial for seed management. Compared with manual data logging and seed management, the data tracking of the facility made data management and seed management more robust and reliable.

Conclusions

This paper described an engineering prototype for the automatic evaluation of rice yield traits, including the number of total spikelets, the number of filled spikelets, the grain length, the grain width, and the 1000-grain weight. The prototype comprised three major units: the threshing unit, the inspection unit, and the packing-weighing unit. The mean absolute percentage error was less than 5% for all of the evaluated yield traits, and the efficiency was approximately 1440 plants per 24-hours continuous workday. The facility will be helpful for improving the accuracy and efficiency of rice yield trait evaluation and will serve as a powerful tool in rice plant phenotyping, which will eventually benefit rice breeding, genetic research, functional genomics research and other rice research. With some modifications, the application could be extended and generalized to other crops, such as wheat, corn and barley. Other compound yield traits such as the seed setting rate and the length-width ratio can also be deduced from the extracted traits. In summary, using agricultural photonics, the high-throughput facility, dubbed the Seed-Evaluation Accelerator, gives plant scientists a novel tool to unlock the phenotypic information coded in rice genome [25].

Methods

Image acquisition

The control software was designed for evaluating yield traits of one rice plant at a time. The continuous acquisition of the images was controlled by the pulses received from a photoelectric sensor attached to the panicle inlet of the threshing unit. Images were acquired and stored using the NI-IMAQ Virtual Instruments (VI) Library for LabVIEW (National Instruments Corporation, USA). For each plant, 14 images were acquired by the "total-spikelet-vision camera" (called the total-spikelet image) and 20 images were acquired by the "filled-spikelet-vision camera" (called the filled-spikelet image). This design was applicable to most rice varieties with less than 20 panicles per plant. Figure 12 shows typical total-spikelet and filled-spikelet images acquired by the two cameras. Note that the images shown in Figure 12 have been cropped for better visualization, as the original images are too large (5000 × 2048 pixels).
Figure 12

Typical grayscale images for (a) a total-spikelet image and (b) a filled- spikelet image.

Image analysis

The image-analysis software was programmed using NI Vision for LabVIEW 8.6 (National Instruments Corporation, USA). The software was designed to allow the two cameras (the "total-spikelet-vision camera" and the "filled-spikelet-vision camera") to work at the same time, and the images were analyzed in the computer simultaneously while the cameras were acquiring new images, thus optimizing the measurement efficiency.

Figure 13 outlines the flowchart of the algorithm for determining the spikelet number in an image (a total-spikelet image or a filled-spikelet image). Image segmentation was performed to determine the background and objects of interest. For better processing speed, a pixel-oriented segmentation algorithm was applied in this study. A pixel was considered to belong to a background point if its grayscale fell below a pre-defined, fixed threshold. In a subsequent step, a median filter (3 × 3 neighborhood) was used to remove isolated pixels. As some small pieces of branches may appear on the conveyor, objects with a length-width ratio greater than three times that of the spikelets were treated as branches and removed. Branches with a length-width ratio less than three times that of the spikelets were subsequently removed using the "IMAQ detect line" operation. Small regions with an area less than half of the average area of spikelets were regarded as impurities and removed from the image. Spikelets may be touching during on-line processing. The shape of an isolated spikelet is roughly elliptical, so the "IMAQ detect Ellipse" operation was used to identify the isolated spikelet in the image (NI Vision Concepts Manual, National Instruments Corporation, USA). After identification, the original image was divided into two images, one image with only the isolated spikelet (isolated image) and the other with only the touching spikelets (touching image). From an efficiency perspective, we opted for a simple area-determination method to determine the spikelet number in the touching image N touching , which was computed by Eq. 5 and Eq. 6:
Figure 13

Determination of spikelet number in an image.

N j = round ( A j A ¯ )
(5)
N t o u c h i n g = j = 1 T N j
(6)

where N j was the actual spikelet number for a given touching region j in the touching image, the function round(x) rounded x to the nearest integer, A j was the area of the touching region j, Ā was the average spikelet area calculated from the isolated image, and T was the number of touching regions.

The spikelet number (N) in the original image was determined by summing up the spikelet numbers in the isolated image (N isolated ) and in the touching image (N touching ). The NTS of the evaluated plant was calculated as the sum of the spikelet numbers in all 14 total-spikelet images. Similarly, the NFS of the evaluated plant was calculated as the sum of the spikelet numbers in all 20 filled-spikelet images.

The length and width of each isolated spikelet in the filled-spikelet images were calculated. The GL is defined as the maximum Euclidean distance between two boundary points of a filled spikelet, and the GW is defined as the maximum length of straight lines perpendicular to the line of the GL.

Figure 14 shows the structure of the image-processing program in LabVIEW for total-spikelet images and filled-spikelet images. Illustrations showing block diagrams of the VIs developed in this research are attached in Additional files 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17.
Figure 14

Structure of image processing program in LabVIEW for (a) total-spikelet images and (b) filled-spikelet images. 1: Source code file 1 (ImProcessPerPlant_TotalSpikeletImage.vi), 2: Source code file 2 (ImageAnalysis_TotalSpikeletImage.vi), 3: Source code file 3 (ImagePreProcess.vi), 4: Source code file 4 (Merge.vi), 5: Source code file 5(Split.vi), 6: Source code file 6 (ImageProcess.vi), 7: Source code file 7 (ImpurityRemove.vi), 8: Source code file 8 (StemRemove.vi), 9: Source code file 9 (MeanAreaCalculation.vi), 10: Source code file 10 (RemoveSmallParticle.vi), 11: Source code file 11 (GrainClassification.vi), 12: Source code file 12 (TotalGrainNumberCalculation.vi), 13: Source code file 13 (TouchingGrainNumberCalculation.vi), 14: Source code file 14 (ImProcessPerPlant_FilledSpikeletImage.vi), 15: Source code file 15 (ImageAnalysis_FilledSpikeletImage.vi), 16: Source code file 16 (GetLengthWidthRatio.vi). Illustrations showing source code files 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16 are attached in Additional files 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17, respectively.

Notes

Declarations

Acknowledgements

The authors deeply appreciate the cooperation of our partners at Huazhong Agricultural University for providing the grain samples. This work was supported the Key Program of the Natural Science Foundation of Hubei Province (Grant No. 2008CDA087) and the Program for New Century Excellent Talents in University (No. NCET-10-0386).

Authors’ Affiliations

(1)
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology
(2)
Key Laboratory of Biomedical Photonics of Ministry of Education, College of Life Science and Technology, Huazhong University of Science and Technology

References

  1. Zhang Q: Strategies for developing green super rice. PNAS. 2007, 104 (42): 16402-16409. 10.1073/pnas.0708013104.PubMed CentralView ArticlePubMedGoogle Scholar
  2. Wang E, Wang J, Zhu X, Hao W, Wang L, Li Q, Zhang L, He W, Lu B, Lin H, Ma H, Zhang G, He Z: Control of rice grain-filling and yield by a gene with a potential signature of domestication. Nature Genetics. 2008, 40 (11): 1370-1374. 10.1038/ng.220.View ArticlePubMedGoogle Scholar
  3. Xing Y, Zhang Q: Genetic and molecular bases of rice yield. Annual Review of Plant Biology. 2010, 61: 11.1-11.22.View ArticleGoogle Scholar
  4. Prasertsak A, Fukai S: Nitrogen availability and water stress interaction on rice growth and yield. Field Crops Research. 1997, 52: 249-260. 10.1016/S0378-4290(97)00016-6.View ArticleGoogle Scholar
  5. Xiao J, Li J, Grandillo S, Ahn SN, Yuan L, Tanksley SD, McCouch SR: Identification of trait-improving quantitative trait loci alleles from a wild rice relative, Oryza rufipogon. Genetics. 1998, 150: 899-909.PubMed CentralPubMedGoogle Scholar
  6. Thomson MJ, Tai TH, McClung AM, Lai XH, Hinga ME, Lobos KB, Xu Y, Martinez CP, McCouch SR: Mapping quantitative trait loci for yield, yield components and morphological traits in an advanced backcross population between Oryza rufipogon and the Oryza sativa cultivar Jefferson. Theoretical and Applied Genetics. 2003, 107: 479-493. 10.1007/s00122-003-1270-8.View ArticlePubMedGoogle Scholar
  7. Bagge M, Lübberstedt T: Functional markers in wheat: technical and economic aspects. Molocular Breeding. 2008, 22: 319-328. 10.1007/s11032-008-9190-6.View ArticleGoogle Scholar
  8. Kolukisaoglu Ü, Thurow K: Future and frontiers of automated screening in plant sciences. Plant Science. 2010, 178: 476-484. 10.1016/j.plantsci.2010.03.006.View ArticleGoogle Scholar
  9. Bylesjö M, Segura V, Soolanayakanahally RY, Rae AM, Trygg J, Gustafsson P, Jansson S, Street NR: LAMINA: a tool for rapid quantification of leaf size and shape parameters. BMC Plant Biology. 2008, 8: 82-90. 10.1186/1471-2229-8-82.PubMed CentralView ArticlePubMedGoogle Scholar
  10. French A, Ubeda-Tomás S, Holman TJ, Bennett MJ, Pridmore T: High-throughput quantification of root growth using a novel image-analysis tool. Plant physiology. 2009, 150: 1784-1795. 10.1104/pp.109.140558.PubMed CentralView ArticlePubMedGoogle Scholar
  11. Yazdanbakhsh N, Fisahn J: High throughput phenotyping of root growth dynamics, lateral root formation, root architecture and root hair development enabled by PlaRoM. Functional Plant Biology. 2009, 36: 938-946. 10.1071/FP09167.View ArticleGoogle Scholar
  12. Wang L, Uilecan IV, Assadi AH, Kozmik CA, Spalding EP: HYPOTrace: image analysis software for measuring hypocotyl growth and shape demonstrated on Arabidopsis seedlings undergoing photomorphogenesis. Plant physiology. 2009, 149: 1632-1637. 10.1104/pp.108.134072.PubMed CentralView ArticlePubMedGoogle Scholar
  13. Yang W, Xu X, Duan L, Luo Q, Chen S, Zeng S, Liu Q: High-throughput measurement of rice tillers using a conveyor equipped with X-ray computed tomography. Review of Scientific Instruments. 2011, 82 (2): 025102-025109. 10.1063/1.3531980.View ArticlePubMedGoogle Scholar
  14. Golzarian MR, Frick RA, Rajendran K, Berger B, Roy S, Tester M, Lun DS: Accurate inference of shoot biomass from high-throughput images of cereal plants. Plant Methods. 2011, 7: 2-10.1186/1746-4811-7-2.PubMed CentralView ArticlePubMedGoogle Scholar
  15. Granier C, Aguirrezabal L, Chenu K, Cookson SJ, Dauzat M, Hamard P, Thioux JJ, Rolland G, Bouchier-Combaud S, Lebaudy A, Muller B, Simonneau T, Tardieu F: PHENOPSIS, an automated platform for reproducible phenotyping of plant responses to soil water deficit in Arabidopsis thaliana permitted the identification of an accession with low sensitivity to soil water deficit. New Phytologist. 2006, 169: 623-635. 10.1111/j.1469-8137.2005.01609.x.View ArticlePubMedGoogle Scholar
  16. Reuzeau C, Pen J, Frankard V, Wolf J, Peerbolte R, Broekaert W, Camp W: TraitMill: a discovery engine for identifying yield-enhancement genes in Cereals. Molecular Plant Breeding. 2005, 3 (5): 753-759.Google Scholar
  17. The Plant Accelerator. [http://www.plantaccelerator.org.au/]
  18. The High Resolution Plan Phenotyping Centre. [http://www.plantphenomics.org.au/HRPPC]
  19. The Leibniz Institute of Plant Genetics and Crop Plant Research (IPK). [http://www.ipk-gatersleben.de]
  20. Institute of Biological, Environmental and Rural Sciences (IBERS). [http://www.aber.ac.uk/en/ibers/]
  21. French National Institute for Agricultural Research (INRA). [http://www.international.inra.fr/]
  22. Duan L, Yang W, Bi K, Chen S, Luo Q, Liu Q: Fast discrimination and counting of filled/unfilled rice spikelets based on bi-modal imaging. Computers and Electronics in Agriculture. 2011, 75: 196-203. 10.1016/j.compag.2010.11.004.View ArticleGoogle Scholar
  23. Igathinathane C, Pordesimo LO, Batchelor WD: Major orthogonal dimensions measurement of food grains by machine vision using ImageJ. Food Research International. 2009, 42: 76-84. 10.1016/j.foodres.2008.08.013.View ArticleGoogle Scholar
  24. Igathinathane C, Pordesimo LO, Columbus EP, Batchelor WD, Methuku SR: Shape identification and particles size distribution from basic shape parameters using ImageJ. Computers and Electronics In Agriculture. 2008, 63: 168-182. 10.1016/j.compag.2008.02.007.View ArticleGoogle Scholar
  25. Finkel E: With 'Phenomics,' plant scientists hope to shift breeding into overdrive. Science. 2009, 325: 380-381. 10.1126/science.325_380.View ArticlePubMedGoogle Scholar

Copyright

© Duan et al; licensee BioMed Central Ltd. 2011

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.