13th Community Wide Experiment on the
Critical Assessment of Techniques for Protein Structure Prediction
`
TS Analysis : Group performance based on combined z-scores
Results Home Table Browser Estimate of Model Accuracy Results RR Assessment Results
The cummulative z-scores in this table are calculated according to the following procedure (example for the "first" models):
1. Calculate z-scores from the raw scores for all "first" models (corresponding values from the main result table);
2. Remove outliers - models with zscores below the tolerance threshold (set to -2.0);
3. Recalculate z-scores on the reduced dataset;
4. Assign z-scores below the penalty threshold (either -2.0 or 0.0) to the value of this threshold.
  GDT_TS based   Assessors' formula
    Models:
    • Analysis on the models designated as "1"
    • Analysis on the models with the best scores
  • Groups:
    • All groups on 'all groups' targets
    • Server groups on 'all groups' + 'server only' targets
  • Targets:
      The ranking of the groups is performed on the basis of analysis of combined zscores according to the following formula used by the assessors:
    • for TBM domains (GDT_HA + (SG + lDDT + CAD)/3 + ASE)
    • for TBM + TBM/FM domains (GDT_HA + (SG + lDDT + CAD)/3 + ASE)

    • for FM + TBM/FM domains (GDT_TS + QCS)
    • for FM domains (GDT_TS + QCS)
    #     GR
    code
    GR
    name
    Domains Count     SUM Zscore
    (>-2.0)
    Rank SUM Zscore
    (>-2.0)
    AVG Zscore
    (>-2.0)
    Rank AVG Zscore
    (>-2.0)
    SUM Zscore
    (>0.0)
    Rank SUM Zscore
    (>0.0)
    AVG Zscore
    (>0.0)
    Rank AVG Zscore
    (>0.0)
1 043 A7D 31 54.9935 1 1.7740 1 54.9935 1 1.7740 1
2 322 Zhang 31 42.7836 2 1.3801 2 42.8133 2 1.3811 2
3 145 QUARK 31 36.2191 3 1.1684 3 36.2827 4 1.1704 4
4 089 MULTICOM 31 36.1782 4 1.1670 4 36.5381 3 1.1786 3
5 261 Zhang-Server 31 33.2515 5 1.0726 5 33.4101 5 1.0777 5
6 224 Destini 31 31.9069 6 1.0293 6 32.7036 6 1.0550 6
7 498 RaptorX-Contact 31 29.6487 7 0.9564 7 30.7070 7 0.9905 8
8 197 MESHI 31 27.7536 8 0.8953 9 29.0389 9 0.9367 10
9 354 wfAll-Cheng 31 27.4311 9 0.8849 10 29.8221 8 0.9620 9
10 196 Grudinin 31 27.4111 10 0.8842 11 28.5938 10 0.9224 11
11 324 RaptorX-DeepModeller 31 26.9495 11 0.8693 12 27.6441 11 0.8917 12
12 117 Jones-UCL 31 25.9528 12 0.8372 13 26.7971 13 0.8644 14
13 460 McGuffin 31 25.5098 13 0.8229 14 26.5964 14 0.8579 15
14 135 SBROD 31 25.2040 14 0.8130 15 27.5029 12 0.8872 13
15 208 KIAS-Gdansk 31 23.7414 15 0.7659 16 26.2444 15 0.8466 16
16 055 VoroMQA-select 31 22.7581 16 0.7341 17 25.5219 17 0.8233 18
17 418 Seder3nc 31 22.1343 17 0.7140 18 25.5574 16 0.8244 17
18 457 Wallner 31 20.1724 18 0.6507 19 22.3701 22 0.7216 23
19 192 Elofsson 31 20.1500 19 0.6500 20 23.6015 18 0.7613 19
20 044 ProQ2 31 19.7657 20 0.6376 21 23.0521 19 0.7436 20
21 274 MUFold 31 19.2989 21 0.6225 22 22.5383 20 0.7270 21
22 086 BAKER 31 19.1017 22 0.6162 23 22.0457 24 0.7112 25
23 426 AP_1 31 18.8315 23 0.6075 24 20.9985 27 0.6774 29
24 406 Seder3mm 31 18.0984 24 0.5838 25 21.9546 25 0.7082 26
25 222 Seok-refine 31 17.8608 25 0.5762 26 21.7174 26 0.7006 27
26 309 Seder1 31 17.5873 26 0.5673 27 20.8672 28 0.6731 30
27 071 Seder3full 31 16.0918 27 0.5191 28 22.4634 21 0.7246 22
28 344 Kiharalab 31 15.5561 28 0.5018 30 17.4428 30 0.5627 32
29 221 RaptorX-TBM 31 12.7250 29 0.4105 32 16.8524 32 0.5436 34
30 241 Bhageerath-Star 31 11.3868 30 0.3673 34 15.9029 34 0.5130 35
31 368 BAKER-ROSETTASERVER 31 9.9346 31 0.3205 35 14.9780 35 0.4832 37
32 279 ZHOU-SPOT 30 9.7590 32 0.3920 33 16.4957 33 0.5499 33
33 390 Bhattacharya 31 4.9697 33 0.1603 37 11.2449 38 0.3627 41
34 149 Zhou-SPOT-3D 31 4.9322 34 0.1591 38 10.2552 41 0.3308 43
35 214 wfRosetta-ModF7 29 3.5453 35 0.2602 36 14.2723 36 0.4921 36
36 244 Seder3hard 31 2.8074 36 0.0906 42 17.7844 29 0.5737 31
37 366 Venclovas 22 2.3227 37 0.9238 8 22.3122 23 1.0142 7
38 492 wf-BAKER-UNRES 31 1.2545 38 0.0405 43 7.6551 47 0.2469 49
39 116 Zhang-CEthreader 31 -0.2568 39 -0.0083 44 7.7950 46 0.2515 48
40 068 Seok 31 -0.8171 40 -0.0264 46 11.9374 37 0.3851 38
41 124 AWSEM-Suite 31 -0.8835 41 -0.0285 47 7.0688 48 0.2280 50
42 164 Yang-Server 30 -2.2901 42 -0.0097 45 9.2665 44 0.3089 45
43 058 MULTICOM_CLUSTER 31 -2.4593 43 -0.0793 49 6.5081 49 0.2099 51
44 243 MULTICOM-CONSTRUCT 31 -2.6228 44 -0.0846 50 6.4633 50 0.2085 52
45 163 Bates_BMM 24 -3.3523 45 0.4437 31 17.2261 31 0.7178 24
46 377 wfRstta-Maghrabi-TQA 31 -3.9039 46 -0.1259 51 9.5324 43 0.3075 46
47 329 D-Haven 30 -3.9353 47 -0.0645 48 10.4018 40 0.3467 42
48 023 MULTICOM-NOVEL 31 -4.1631 48 -0.1343 53 4.6844 55 0.1511 56
49 335 wfRosetta-PQ2-AngQA 31 -4.8070 49 -0.1551 54 10.1583 42 0.3277 44
50 351 DL-Haven 31 -4.9674 50 -0.1602 55 4.9423 54 0.1594 55
51 112 AWSEM 30 -5.8371 51 -0.1279 52 5.7157 52 0.1905 53
52 288 UNRES 31 -11.2067 52 -0.3615 57 5.2662 53 0.1699 54
53 152 PconsC4 31 -11.5002 53 -0.3710 58 4.6317 56 0.1494 57
54 381 GONGLAB-THU 31 -12.5245 54 -0.4040 59 1.2163 73 0.0392 74
55 160 CMA-align 31 -12.7678 55 -0.4119 60 3.2276 59 0.1041 62
56 441 FALCON 31 -12.9039 56 -0.4163 61 4.0852 57 0.1318 59
57 246 IntFOLD5 31 -13.0352 57 -0.4205 62 3.1277 60 0.1009 64
58 281 SHORTLE 23 -13.0570 58 0.1280 40 8.6968 45 0.3781 40
59 122 Forbidden 27 -15.1294 59 -0.2641 56 3.3550 58 0.1243 60
60 407 rawMSA 30 -16.5691 60 -0.4856 63 1.3682 71 0.0456 72
61 041 FALCON-Contact 31 -17.1976 61 -0.5548 67 3.0365 61 0.0980 65
62 471 CPClab 31 -17.4853 62 -0.5640 68 2.5261 66 0.0815 67
63 337 FALCON-TBM 31 -17.9825 63 -0.5801 69 1.7893 68 0.0577 69
64 110 Distill 31 -21.1311 64 -0.6816 73 0.1358 83 0.0044 85
65 488 Delta-Gelly-Server 31 -21.2291 65 -0.6848 75 2.0065 67 0.0647 68
66 282 PRAYOG 27 -22.1180 66 -0.5229 65 0.7234 74 0.0268 77
67 047 chuo-u 30 -22.4647 67 -0.6822 74 0.2674 80 0.0089 84
68 156 Seok-server 31 -23.0087 68 -0.7422 77 2.6212 65 0.0846 66
69 358 Spider 31 -23.4856 69 -0.7576 78 0.6094 75 0.0197 80
70 446 slbio 15 -24.3085 70 0.5128 29 10.4461 39 0.6964 28
71 402 RBO-Aleph 29 -24.7846 71 -0.7167 76 2.9437 62 0.1015 63
72 348 HMSCasper-Refiner 31 -25.5918 72 -0.8255 80 0.0043 87 0.0001 88
73 085 BhageerathH-Plus 31 -26.4686 73 -0.8538 83 0.0510 86 0.0016 86
74 497 GaussDCA 31 -27.1931 74 -0.8772 85 1.6783 69 0.0541 70
75 431 Laufer 16 -27.8307 75 0.1356 39 6.0986 51 0.3812 39
76 414 BCLMeilerLab 30 -29.2032 76 -0.9068 86 0.0000 89 0.0000 89
77 004 YASARA 27 -29.5182 77 -0.7970 79 0.4806 78 0.0178 81
78 312 MUFold_server 31 -31.7040 78 -1.0227 89 0.2906 79 0.0094 83
79 470 Seok-assembly 24 -34.4328 79 -0.8514 82 2.8672 63 0.1195 61
80 378 Cao-server 31 -34.5740 80 -1.1153 92 0.0000 89 0.0000 89
81 266 slbio_server 28 -34.6782 81 -1.0242 90 1.4447 70 0.0516 71
82 472 DELClab 30 -37.7127 82 -1.1904 93 0.0000 89 0.0000 89
83 365 3D-JIGSAW_SL1 31 -39.0676 83 -1.2602 94 0.0000 89 0.0000 89
84 157 GAPF_LNCC 17 -39.4029 84 -0.6708 72 0.5244 77 0.0308 76
85 007 ACOMPMOD 31 -41.0460 85 -1.3241 95 0.0000 89 0.0000 89
86 458 FOLDNET 23 -41.2638 86 -1.0984 91 0.0000 89 0.0000 89
87 097 Laufer_abinitio 13 -42.5878 87 -0.5068 64 0.5417 76 0.0417 73
88 257 NOCONTACT 31 -42.7573 88 -1.3793 96 0.0000 89 0.0000 89
89 077 qmo 9 -43.0897 89 0.1011 41 2.6253 64 0.2917 47
90 401 InnoUNRES 12 -49.2542 90 -0.9379 88 0.2484 81 0.0207 78
91 432 Seok-naive_assembly 10 -50.6660 91 -0.8666 84 1.3678 72 0.1368 58
92 473 Maurice 7 -52.5862 92 -0.6552 71 0.0780 85 0.0111 82
93 476 Sun_Tsinghua 17 -53.6425 93 -1.5084 98 0.0000 89 0.0000 89
94 380 Laufer_100 6 -53.7782 94 -0.6297 70 0.1224 84 0.0204 79
95 397 PepBuilderJ 15 -53.8033 95 -1.4536 97 0.0000 89 0.0000 89
96 389 UpsideUChicago 6 -55.0712 96 -0.8452 81 0.1944 82 0.0324 75
97 347 MESHI-server 3 -57.6137 97 -0.5379 66 0.0027 88 0.0009 87
98 092 Ricardo 1 -60.9136 98 -0.9136 87 0.0000 89 0.0000 89
Protein Structure Prediction Center
Sponsored by the US National Institute of General Medical Sciences (NIH/NIGMS)
Please address any questions or queries to:
© 2007-2018, University of California, Davis