Skip to main content

Table 1 Measurement of agreement between experienced and non-experienced observers

From: Citizen crowds and experts: observer variability in image-based plant phenotyping

 

DiC ↓

|DiC| ↓

MSE ↓

R2↑

Alpha ↑

Intra-observer (RPi) tool

Experienced [The reference observer]a

0.10 (0.54)

0.29 (0.47)

0.307

0.980

0.987

Non-experienced

0.13 (0.77)

0.42 (0.65)

0.600

0.960

0.981

Tool versus visual (RPi)

Experienced

0.00 (0.64)

0.33 (0.55)

0.415

0.970

0.986

Non-experienced

0.23 (0.82)

0.46 (0.71)

0.730

0.950

0.977

Inter-observer (RPi) tool

Experienced

0.07 (0.65)

0.37 (0.53)

0.423

0.974

0.980

Non-experienced

0.49 (0.76)

0.60 (0.67)

0.815

0.962

0.962

Inter-observer (Canon) tool

Experienced

0.55 (0.74)

0.63 (0.68)

0.861

0.969

0.959

Non-experienced

0.23 (0.63)

0.37 (0.56)

0.450

0.977

0.976

Intra-observer across resolution (RPi and Canon) tool

Experienced

0.57 (0.87)

0.68 (0.79)

1.100

0.950

0.965

Non-experienced

0.40 (0.70)

0.51 (0.62)

0.650

0.973

0.977

Citizens inter-observer (RPi) zooniverse

Experienced versus consensus (average)

0.53 (0.77)

0.62 (0.69)

0.869

0.962

0.960

Experienced versus consensus (max)

0.08 (0.82)

0.45 (0.69)

0.684

0.957

0.971

Consensus (average) versus sing. random

0.00 (0.78)

0.42 (0.65)

0.607

0.960

0.970

  1. For shorthand definitions see text. For DiC and |DiC| average and standard deviation are reported. Note that these correspond also to bias and limits of agreement (when standard deviation is multiplied by 1.96) of the Bland–Altman plots reported. \(\downarrow\) means lower is better, whereas \(\uparrow\) the opposite
  2. aThis experienced observer is noted and used as the reference observer for the remaining analysis throughout the paper