15th Community Wide Experiment on the
Critical Assessment of Techniques for Protein Structure Prediction
Menu
Multimer Predictions Analysis : Group performance based on combined z-scores
Results Home Table Browser
  • Analysis on the models designated as "1"
  • Analysis on the models with the best scores
Targets :
  • TBM (easy)
  • TBM/FM (medium)
  • FM (hard)
  • X-ray
  • NMR
  • CryoEM
Groups:
  • CASP groups on all targets
  • CASP+CAPRI groups on CAPRI targets only

Assessors' formula: Z-score(ICS) + Z-score(IPS) + Z-score(LDDTo) + Z-score(TM)

*ICS: Interface Contact Score (a.k.a. F1 score)
*IPS: Interface Patch Score (a.k.a. Jaccard coefficient)

** The performance graph and table were updated on May 3, 2023 to exlude two targets with unreliable experimental stoichiometry data. Originally, results for 43 targets were assessed. This change had marginal effect on the cumulative Z-scores and did not affect the group ranking reported at the CASP15 meeting.

    #     GR
    name		     GR
    code		      Targets
     Count		     SUM Zscore
    (>0.0)		     Rank SUM
    Zscore (>0.0)		     AVG Zscore
    (>0.0)		     Rank AVG 
    Zscore  (>0.0)		     SUM Zscore
    (>-2.0)		     Rank SUM 
    Zscore  (>-2.0)		     AVG Zscore
    (>-2.0)		     Rank AVG 
    Zscore  (>-2.0)
1 Zheng 374 41 35.2994 1 0.8610 1 33.6645 1 0.8211 1
2 Venclovas 494 41 29.1459 2 0.7109 4 24.6965 2 0.6024 5
3 Wallner 037 35 28.1387 3 0.8040 2 9.3089 15 0.6088 4
4 Yang-Multimer 239 39 24.6946 4 0.6332 7 14.1477 6 0.4653 7
5 Yang 439 39 24.1739 5 0.6198 8 14.0841 7 0.4637 8
6 Kiharalab 119 41 21.8234 6 0.5323 13 13.1414 9 0.3205 17
7 MULTICOM_human 003 41 20.7152 7 0.5052 15 17.5913 3 0.4291 10
8 Manifold 248 41 20.2878 8 0.4948 17 10.8563 12 0.2648 23
9 McGuffin 180 41 19.8916 9 0.4852 18 13.6438 8 0.3328 16
10 MULTICOM 367 41 19.6505 10 0.4793 20 15.6138 4 0.3808 12
11 Manifold-E 035 41 18.8589 11 0.4600 22 9.6659 14 0.2358 25
12 MULTICOM_qa 086 41 18.3529 12 0.4476 24 14.3743 5 0.3506 13
13 PEZYFoldings 278 39 17.9454 13 0.4601 21 3.1746 23 0.1840 29
14 DFolding-server 288 33 17.0135 14 0.5156 14 -8.4155 31 0.2298 27
15 MULTICOM_deep 158 41 16.2869 15 0.3972 29 11.3980 11 0.2780 22
16 CoDock 444 41 16.2632 16 0.3967 30 1.4193 24 0.0346 45
17 BAKER 185 41 15.9119 17 0.3881 32 4.7296 22 0.1154 40
18 UltraFold 054 41 15.7797 18 0.3849 33 7.0504 17 0.1720 33
19 BeijingAIProtein 399 40 15.7162 19 0.3929 31 5.1038 21 0.1776 31
20 UltraFold_Server 125 41 15.7081 20 0.3831 35 6.9314 18 0.1691 34
21 Elofsson 320 41 15.6869 21 0.3826 36 11.7546 10 0.2867 21
22 Takeda-Shitaka_Lab 348 41 15.6274 22 0.3812 37 10.4104 13 0.2539 24
23 MultiFOLD 462 41 15.2358 23 0.3716 38 1.1839 25 0.0289 46
24 MUFold_H 360 41 15.0965 24 0.3682 40 8.7050 16 0.2123 28
25 colabfold_human 461 40 14.3436 25 0.3586 41 -0.4850 26 0.0379 44
26 MUFold 298 41 14.0905 26 0.3437 42 6.7627 19 0.1649 35
27 Pierce 314 26 14.0436 27 0.5401 12 -25.4575 41 0.1747 32
28 Kiharalab_Server 131 40 13.5184 28 0.3380 44 -7.0835 30 -0.1271 52
29 ColabFold 446 39 12.7694 29 0.3274 46 -4.7678 29 -0.0197 48
30 NBIS-AF2-multimer 390 41 12.2710 30 0.2993 49 5.3371 20 0.1302 39
31 RaptorX-Multimer 071 40 11.9178 31 0.2979 50 -2.0280 28 -0.0007 47
32 Grudinin 150 37 11.8864 32 0.3213 48 -1.2193 27 0.1833 30
33 DMP 477 31 11.4191 33 0.3684 39 -18.2551 36 0.0563 43
34 Yang-Server 229 21 10.4950 34 0.4998 16 -32.8774 46 0.3392 14
35 SHT 147 40 10.2014 35 0.2550 57 -18.9473 37 -0.4237 67
36 DFolding-refine 073 35 9.3295 36 0.2666 55 -26.8823 42 -0.4252 68
37 ShanghaiTech 225 22 9.2790 37 0.4218 25 -34.8589 47 0.1428 37
38 ClusPro 350 36 9.0379 38 0.2511 59 -12.7073 33 -0.0752 50
39 GuijunLab-Human 169 39 8.8648 39 0.2273 61 -11.5168 32 -0.1927 56
40 Coqualia 434 21 8.6567 40 0.4122 26 -38.0125 51 0.0946 41
41 GuijunLab-Assembly 098 41 8.6000 41 0.2098 63 -13.7916 35 -0.3364 62
42 FTBiot0119 165 41 7.6695 42 0.1871 66 -21.4535 39 -0.5233 71
43 Shen-CAPRI 493 34 7.5883 43 0.2232 62 -28.2620 45 -0.4195 65
44 trComplex 423 38 7.4448 44 0.1959 65 -22.0194 40 -0.4216 66
45 GinobiFold-SER 011 17 6.7751 45 0.3985 28 -49.0828 58 -0.0637 49
46 GuijunLab-DeepDA 188 41 6.4491 46 0.1573 69 -13.2195 34 -0.3224 61
47 TRFold 187 37 6.1285 47 0.1656 68 -27.5451 44 -0.5282 72
48 Zou 205 36 6.1151 48 0.1699 67 -19.6517 38 -0.2681 60
49 GinobiFold 227 21 5.5483 49 0.2642 56 -44.6555 55 -0.2217 57
50 Manifold-X 304 10 5.4396 50 0.5440 11 -58.6086 62 0.3391 15
51 WL_team 257 21 5.3112 51 0.2529 58 -47.5712 56 -0.3605 63
52 DELCLAB 447 35 5.2725 52 0.1506 70 -48.4542 57 -1.0415 79
53 Agemo_mix 092 21 5.1190 53 0.2438 60 -42.4132 54 -0.1149 51
54 Kozakov-Vajda 291 25 5.0204 54 0.2008 64 -35.2959 48 -0.1318 53
55 UNRES 091 37 4.8890 55 0.1321 72 -39.4926 52 -0.8512 76
56 DFolding 074 10 4.8492 56 0.4849 19 -60.6654 63 0.1335 38
57 ShanghaiTech-TS-SER 133 17 4.7210 57 0.2777 51 -50.3994 59 -0.1411 54
58 FoldEver-Hybrid 385 36 4.6375 58 0.1288 73 -27.1271 43 -0.4758 69
59 FoldEver 245 29 4.3176 59 0.1489 71 -35.4943 49 -0.3964 64
60 Manifold-LC-E 046 9 4.0502 60 0.4500 23 -62.6889 64 0.1457 36
61 bio3d 397 6 3.8369 61 0.6395 6 -68.2028 69 0.2995 19
62 Fernandez-Recio 312 32 3.8177 62 0.1193 75 -40.7973 53 -0.7124 75
63 ChaePred 398 34 3.4678 63 0.1020 78 -35.6092 50 -0.6356 74
64 OpenFold 441 25 2.8250 64 0.1130 76 -57.5014 60 -1.0201 77
65 OpenFold-SingleSeq 433 25 2.8250 64 0.1130 76 -57.5014 60 -1.0201 77
66 AIchemy_LIG3 347 10 2.7365 66 0.2737 52 -63.6011 65 -0.1601 55
67 AIchemy_LIG 325 10 2.7082 67 0.2708 53 -64.2274 67 -0.2227 58
68 AIchemy_LIG2 456 10 2.7082 67 0.2708 53 -64.2274 67 -0.2227 58
69 ddquest 472 3 2.2704 69 0.7568 3 -73.8743 77 0.7086 2
70 KORP-PL 352 4 2.1949 70 0.5487 10 -72.1479 74 0.4630 9
71 Convex-PL-R 460 3 2.0276 71 0.6759 5 -73.9724 78 0.6759 3
72 zax 122 6 2.0003 72 0.3334 45 -68.5982 71 0.2336 26
73 UTMB 201 5 1.9189 73 0.3838 34 -70.5262 73 0.2948 20
74 Convex-PL 338 3 1.7158 74 0.5719 9 -74.3057 79 0.5648 6
75 TB_model_prediction 199 4 1.3073 75 0.3268 47 -73.6242 75 0.0939 42
76 XRC_VU 215 14 1.1453 76 0.0818 80 -68.9593 72 -1.0685 80
77 Graphen_Medical 097 10 0.9378 77 0.0938 79 -68.3170 70 -0.6317 73
78 ESM-single-sequence 067 13 0.8883 78 0.0683 82 -73.6771 76 -1.3598 83
79 Gonglab-THU 052 7 0.5344 79 0.0763 81 -75.6460 80 -1.0923 81
80 Cerebra 315 8 0.5344 79 0.0668 83 -75.6460 80 -1.2058 82
81 noxelis 236 1 0.4100 81 0.4100 27 -79.5900 83 0.4100 11
82 TensorLab 132 3 0.3590 82 0.1197 74 -77.4791 82 -0.4930 70
83 Panlab 234 28 0.3532 83 0.0126 84 -64.1904 66 -1.3639 84
84 GuijunLab-Meta 481 1 0.3431 84 0.3431 43 -79.6894 84 0.3106 18
85 FALCON2 368 20 0.0000 85 0.0000 85 -80.2142 85 -1.9107 85
86 FALCON0 333 20 0.0000 85 0.0000 85 -80.2142 85 -1.9107 85
87 wuqi 370 1 0.0000 85 0.0000 85 -82.0000 87 -2.0000 87
The cummulative z-scores in this table are calculated according to the following procedure (example for the "first" models):
1. Calculate zscores from the raw scores for all "first" models (corresponding values from the main result table);
2. Remove outliers - models with zscores below the tolerance threshold (set to -2.0);
3. Recalculate zscores on the reduced dataset;
4. Assign z-scores below the penalty threshold (either -2.0 or 0.0) to the value of this threshold.
Protein Structure Prediction Center
Sponsored by the US National Institute of General Medical Sciences (NIH/NIGMS)
Please address any questions or queries to:
© 2007-2022, University of California, Davis