Table 1 Comparison of different PLMs and different pooling layers
PLM layer | Pooling operations | Pearson (%) | Spearman (%) |
|---|---|---|---|
BERT | First-last avg | 84.84 | 77.22 |
RoBERTa | First-last avg | 84.6 | 77.07 |
MacBERT | First-last avg | 85.66 | 77.10 |
BERT | Mean | 86.11 | 77.53 |
RoBERTa | Mean | 85.80 | 77.29 |
MacBERT | Mean | 85.89 | 77.33 |
BERT | Pooler | 84.40 | 77.02 |
RoBERTa | Pooler | 84.29 | 77.11 |
MacBERT | Pooler | 83.83 | 77.41 |
BERT | CLS | 85.95 | 77.27 |
RoBERTa | CLS | 85.95 | 77.16 |
MacBERT | CLS | 86.34 | 77.67 |