16th Community Wide Experiment on the
Critical Assessment of Techniques for Protein Structure Prediction
CASP16
CASP provides an independent mechanism for the assessment of methods of protein structure modeling. From May through August 2024, CASP organizers have been posting on this website sequences of unknown protein structures for modeling. Protein models have been collected from May through mid-September, and evaluated as the experimental coordinates become available. In the summer and fall, the tens of thousands of models submitted by approximately 100 research groups worldwide are processed and evaluated. Independent assessors in each of the prediction categories bring independent insight into their assessment. Tools for viewing, comparison, and analysis of submitted models are available from this website.

Targets Predictors Conference Results
Target List
Domain Definition
Groups Info
Register for the conference
Program
Abstracts
Presentations
PROTEIN DOMAINS
PROTEIN MULTIMERS
RNA/DNA
RNA/DNA MULTIMERS
EMA (ACCURACY)
LIGAND
 
Parseable Data
 
Rankings: Protein Domains
Rankings: Protein multimers
Rankings: RNA/DNA monomers
Rankings: RNA/DNA multimers
Rankings: EMA (accuracy)
Rankings: Ligands

CASP16 in numbers

Detailed description of the experiment

CASP (Critical Assessment of Structure Prediction) is a community wide experiment to determine and advance the state of the art in computational structural biology. Every two years, participants are invited to submit models for a set of macromolecules and macromolecular complexes (proteins, RNA, ligands) for which the experimental structures are not yet public. In the latest CASP round, CASP16 in 2024, nearly 100 research groups from around the world submitted more than 80,000 models on 100+ modeling entities yielding 300 targets in five prediction categories. Independent assessors then compare the models with experiment. Assessments and results are published in a special issue of the journal PROTEINS (check the latest CASP15 issue here).

Goals Categories Timetable Registration Targets Format Assessment Results Conference Organizers

Background and goals

The goals of CASP are to provide rigorous assessment of computational methods for modeling macromolecular structures and complexes so as to advance the state of the art. Recent CASPs saw enormous jumps in the accuracy of computed structures, first in CASP14 (2020) for single proteins and domains, with many models competitive in accuracy with experiment, and second, in CASP15 (2022), with a large increase in the accuracy of protein complexes. These advances are primarily the result of the successful application of deep learning methods, particularly AlphaFold2 and other methods built around it. Major interest in the field is now centered around the further potential impact of deep learning methods. In response to that, in the 2022 CASP15, modeling categories were realigned. We are maintaining those categories in CASP16. We will also continue our close collaborations with CAPRI (for protein complexes) and RNA puzzles (for RNA structure).

Modeling categories

CASP16 categories are as follows:
  • Single Proteins and Domains
    As in previous CASPs, the accuracy of single proteins and where appropriate single protein domains will be assessed, using the established metrics. The major emphasis is now on the fine-grained accuracy of models, whether limitations related to sequence alignment depth and target size are surmounted, and whether interdomain relationships are accurately captured. There is also interest how well the many new deep learning methods perform, including those using large language models.
  • Protein Complexes
    As in recent CASPs, the ability of current methods to correctly model subunit-subunit and protein-protein interactions will be assessed. We will again work in close collaboration with our CAPRI partners. There was enormous progress in this category in the last CASP, but accuracy was not yet as high as for single proteins, so there is substantial room for a further advance. New in this CASP is the option of predicting stoichiometry. Where possible, targets will initially be released without that information, models collected, followed by re-release with that data provided.
  • Accuracy Estimation
    Members of the community will again be invited to submit accuracy estimates for multimeric complexes and inter-subunit interfaces provided by others. There is no longer a category for general methods of estimating single protein structure accuracy, since in recent CASPs estimates provided by model builders have been consistently more reliable. However, there will be an emphasis on the reliability of accuracy estimates provided with submitted structures, both overall and at the individual amino acid level. Note that all accuracy estimates are in plddt units, not Angstroms.
  • Nucleic acid (NA) structures and complexes
    An RNA structure category was introduced in the previous CASP and the results were interesting and provocative. In particular, it appeared that deep learning methods were not yet as effective as more traditional ones for this type of macromolecule. Has that now changed? This CASP we expect to include RNA and DNA single structures and complexes, and complexes of these with proteins.
  • Protein - organic ligand complexes
    The last round of CASP included this category for the first time. Results indicated that, as with RNA structure, deep learning methods were not yet competitive with more traditional approaches. So there is considerable interest in whether that has now changed. In addition to ligands integral to protein targets, we expect to have several target sets related to drug design.
  • Macromolecular conformational ensembles
    Following the success of deep-learning methods for single structures, it is increasingly important to assess methods for predicting structure ensembles, and CASP included this category for the first time in 2022. While it was clear deep learning methods have considerable potential for generating ensembles, the best procedures are still hotly debated with many new papers appearing. In CASP16, we expect to have a variety of targets for both protein and RNA ensembles.
  • Integrative modeling
    Deep learning methods combined with sparse experimental data such as SAXS and chemical crosslinking are now being used extensively to obtain the structure of large marcomolecular complexes. To assess effectiveness of these approaches, CASP is reintroducing this category of modeling, provided appropriate targets will be available.

    Timetable

    • April 2, 2024 - Start of the registration for CASP16 prediction experiment.
    • April 16, 2024 - Start of the testing of server connectivity ("dry run" for server predictors).
    • May 1, 2024 - Release of the first CASP16 modeling targets.
    • June/July 2024 - Early bird registration for the December CASP16 conference.
    • July 31, 2024 - Last date for releasing targets.
    • August 31, 2024 - End of the modeling season.
    • Early September 2024 - Collection of abstracts describing the methods used in CASP16.
    • August-October 2024 - Evaluation of predictions.
    • November 2024 - Invitations to groups with the most accurate models and the most interesting methods to give talks at the CASP16 conference.
    • November 2024 - Program of the conference finalized.
    • December 2024 - CASP16 Conference (tentatively, December 1-4).

    Registration

    Participation is open to all.

    CASP16 registration opens on April 2, 2024.

    If you are new to CASP and don't have an account with the Prediction Center, you will have to register with the Prediction Center first and only then proceed to CASP16 registration page, which will be available here on April 2, 2024.

    If you already have an account with the Prediction Center, you can go directly to the CASP16 registration page. Please check, though, that your basic registration information is current. If it has changed - please update it through the My Personal Data link from the main Menu.

    Participants with servers are requested to register before April 9, 2024 as we are planning to start checking servers' format and connectivity thereafter.

    Targets

    CASP16 modeling targets will be announced through the Target List page from the main CASP16 webpage.

    The success of CASP is dependent on the generous help of the experimental community in providing targets. As in previous CASPs, protein crystallographers, NMR spectroscopists and cryo-EM scientists are asked to provide details of structures they expect to have made public before September 15, 2024. All types of macromolecular structures may be good modeling targets, but those that are membrane-related and/or complexes, including combinations of protein, RNA, and DNA components, are particularly needed. Immune-related complexes proved challenging last time, and so targets of that type and also viral-host complexes will be informative. For the protein-organic ligand category, sets of drug design-related complexes are most appropriate, but other complexes are also welcome. Success of the macromolecular ensembles category depends on the availability of targets with multiple experimentally observed conformations. The last day for suggesting proteins as CASP targets is July 20, 2024. A target submission form is available here.
    Details on the target collection and release procedures are available at our Q&A page.

    Model submission /format

    Models can be submitted through the Prediction Submission form available from this web site or by the email provided in the CASP16 format page . Please comply with the instructions on submission procedures and format provided there. Server predictions will be made publicly available shortly after the closing of the prediction window for a specific target.

    Assessment of models

    As is the practice in CASP, assessment of the results will be made by the independent assessor teams. Assessment criteria will be based on those previously developed in CASP, but assessors may add new metrics they consider appropriate. Where possible, results will also be evaluated using criteria from the previous CASP, so the effects of any changes in criteria can be appreciated.

    The CASP16 Assessors are as follows:

    • Protein structure prediction (single proteins and protein assemblies) - Nick Grishin and Qian Cong (University of Texas, Southwestern, USA)
    • Model accuracy estimation - Randy Read (University of Cambridge, UK)
    • Ligands - Michael Gilson (University of California, San Diego, CA, USA)
    • RNA - Rhiju Das (Stanford University, CA, USA) and Eric Westhof (Universite de Strasbourg, France)
    • Ensembles - Gaetano Montelione (Rensselaer Polytechnic Institute, Troy, NY, USA)

    Click here for the list of assessors in all CASPs held so far.

    In accordance with CASP policy, assessors cannot take part in the relevant parts of the experiment as predictors. Participants must not contact assessors directly with queries, but rather these should be sent to the email address.

    Results and publication

    All CASP predictions and results of numerical evaluation will be made available through this web site shortly before the meeting. The proceedings will be published in a scientific journal (see publications of previous experiments). All participants will also be required to describe their methods in the abstracts, which will be published at our web site in late October.

    Conference

    The conference to discuss results of the CASP16 experiment is planned to be held in Caribbean in early December 2024 (tentatively, December 1-4 2024, starting at 5 pm on the 1st and ending in the afternoon of the 4th). Registration for the meeting will open in early September.

    Organizing committee

           John Moult, CASP chair and founder; IBBR, University of Maryland, USA
           Krzysztof Fidelis, founder, University of California, Davis, USA
           Andriy Kryshtafovych, University of California, Davis, USA
           Torsten Schwede, University of Basel, Switzerland
           Maya Topf, Centre for Structural Systems Biology, Hamburg, Germany

    Scientific advisory board

           Minkyung Baek, Seoul University, South Korea
           David Baker, University of Washington, USA
           Charlotte Dean, University of Oxford, UK
           Nick Grishin, University of Texas, USA
           Andrzej Joachimiak, Argonne National Lab, USA
           David Jones, University College, London, UK
           John Jumper, Google Deepmind, London, UK

    Funding

                       
CASP16 results
Dear CASPers, For those of you who missed CASP16 conference in person, we want to let you know that our community had a nice and constructive meeting. The results of the conference have been posted ...
To RNA solvent prediction challenge participants
Due to some very interesting results in the ribozyme solvation shell target R1260 this year, we will be taking an unusual step. We are inviting participants in the solvation prediction challenge to jo ...
To predictors of T1200/T1300 targets
Dear Predictors, We are contacting you since you entered the challenge for predicting ensemble-based or distribution-based structures for flexible linker Spa-N Protein A targets 1200 and 1300. The ...
Protein Structure Prediction Center
Sponsored by the US National Institute of General Medical Sciences (NIH/NIGMS)
Please address any questions or queries to:
© 2007-2024, University of California, Davis
Terms of Use