Skip to main content

Table 1 Measurement of agreement between experienced and non-experienced observers

From: Citizen crowds and experts: observer variability in image-based plant phenotyping

  DiC ↓ |DiC| ↓ MSE ↓ R2 Alpha ↑
Intra-observer (RPi) tool
Experienced [The reference observer]a 0.10 (0.54) 0.29 (0.47) 0.307 0.980 0.987
Non-experienced 0.13 (0.77) 0.42 (0.65) 0.600 0.960 0.981
Tool versus visual (RPi)
Experienced 0.00 (0.64) 0.33 (0.55) 0.415 0.970 0.986
Non-experienced 0.23 (0.82) 0.46 (0.71) 0.730 0.950 0.977
Inter-observer (RPi) tool
Experienced 0.07 (0.65) 0.37 (0.53) 0.423 0.974 0.980
Non-experienced 0.49 (0.76) 0.60 (0.67) 0.815 0.962 0.962
Inter-observer (Canon) tool
Experienced 0.55 (0.74) 0.63 (0.68) 0.861 0.969 0.959
Non-experienced 0.23 (0.63) 0.37 (0.56) 0.450 0.977 0.976
Intra-observer across resolution (RPi and Canon) tool
Experienced 0.57 (0.87) 0.68 (0.79) 1.100 0.950 0.965
Non-experienced 0.40 (0.70) 0.51 (0.62) 0.650 0.973 0.977
Citizens inter-observer (RPi) zooniverse
Experienced versus consensus (average) 0.53 (0.77) 0.62 (0.69) 0.869 0.962 0.960
Experienced versus consensus (max) 0.08 (0.82) 0.45 (0.69) 0.684 0.957 0.971
Consensus (average) versus sing. random 0.00 (0.78) 0.42 (0.65) 0.607 0.960 0.970
  1. For shorthand definitions see text. For DiC and |DiC| average and standard deviation are reported. Note that these correspond also to bias and limits of agreement (when standard deviation is multiplied by 1.96) of the Bland–Altman plots reported. \(\downarrow\) means lower is better, whereas \(\uparrow\) the opposite
  2. aThis experienced observer is noted and used as the reference observer for the remaining analysis throughout the paper
\