Detailed description of the experiment
The main goal of CASP is to obtain an in-depth and objective assessment
of our current
abilities and inabilities in the area of protein structure prediction.
To this end, participants will predict as much as possible about a set
of soon to be known
structures. These will be true predictions, not ‘post-dictions’ made on
already known structures.
CASP10 will particularly address the following questions:
-
Are the models produced similar to the corresponding experimental
structure?
- Is the mapping of the target sequence onto the proposed
structure
(i.e. the alignment) correct?
- Have similar structures that a model can be based on been
identified?
- Are comparative models more accurate than can be obtained
by simply
copying the best template?
- Has there been progress from the earlier CASPs?
- What methods are most effective?
- Where can future effort be most productively focused?
Tertiary structure predictions (TS):
-
The Template Based Modeling (TBM) category will include domains where a
suitable template can be identified that covers all or nearly all of
the target.
-
The Template free modeling (FM) category will include models of proteins
for which no suitable template can be identified.
-
The Refinement category will include selected targets from among
those released in the main modeling experiment to analyze success in refining
models beyond the quality obtained by simply copying from a single template.
For suitable CASP10 targets, we will select one of the best models received during
the prediction season, and reissue it as a starting structure for refinement.
-
The contact-assisted structure modeling category will show how the knowledge
of a few (usually 3 to 5) long-range contacts influences the ability of predictors to
model the complete structure. This experiemnt will be carried out only for the more
challenging CASP10 targets where we can get coordinates in advance and have at least
two weeks for re-prediction.
-
The chemical shifts guided modeling of NMR structures will be performed
on the selected CASP10 targets, for which we can get a chemical shifts table from
the NMR-spectroscopists at least two weeks ahead of public release of the target.
-
The structure modeling based on molecular replacement with ab initio models
and crystallographic diffraction data will be carried out for selected targets
provided we get the structure factors from the crystallographers.
Other prediction categories:
- Detecting residue-residue contacts in proteins (RR).
- Identifying disordered regions in target proteins (DR).
- Function prediction (prediction of binding sites) (FN).
- Quality assessment of models in general (without
knowing native structures) and the reliability of predicting certain
residues in particular (QA).
There will be additional activities included in or related to CASP10,
which extend its scope.
Rolling CASP
will run in parallel with CASP10 in May-July.
We will discuss the results of the CASP ROLL experiment
at the CASP10 predictors' meeting.
FORCASP:
There will be discussion of predictions and methods
on our
FORCASP forum.
Registration for CASP10 will start in the last week of March 2012.
Testing of server connectivity ("dry run" for server predictors) will
be conducted starting April 16, 2012. The first prediction targets will be
released not earlier than May 1; the last prediction targets will be
released not later than July 17; prediction season will end not later
than July 31. Refinement experiment will end not later than August 17.
Abstracts describing the methods tested in CASP10 will be collected in
September. At the same time we will open registration for the meeting.
The program of the meeting will be available in November.
The meeting will take place on December 9-12, and approximately one
month before that groups with the most accurate predictions and interesting
methods will receive invitations to give talks.
Participation is open to all. If you already have an account with the
Prediction Center, you will be able to go directly to the
CASP10 registration page.
Please check, though, that your basic registration information is
current. If it has changed - please update it through the My Personal
Data link from the main Menu. If you are new to CASP and don't have
an account with us, you will have to register with the Prediction
Center first, and only then - for CASP10. Separate
registration forms
for different types of registration are available through this
website. Predictors with servers are requested to register as soon as
possible
as we are planning on starting a
"dry run" for servers in the second decade of April.
For convenience of the predictors who currently participate in CASP ROLL and
plan to participate in CASP10, we automatically enroll them in CASP10 with the
CASP ROLL group id, group name and PIN number. If you are a CASP ROLL participant
but do not plan to take part in CASP10 - please send us a message
and we will skip your automatic CASP10 enrollment. After the automatic registration,
the current CASP ROLL participants will be able to access their registration data
and make updates if necessary (e.g., add information that a server already enrolled
in CASP ROLL will also be sending QA or DR predictions in CASP10). Please note that
you will not be allowed to change your group name for CASP10; if this is imperative
for you - you will have to register a new group with the desired name in CASP10 and
inform us that you don't want your CASP ROLL group enrolled in CASP10. In this case
you would have to handle submissions separately for the groups registered for different
experiments.
For the experiment to succeed, it is essential that we obtain the help
of the experimental
community. As in previous CASPs, we invite protein crystallographers and NMR
spectroscopists to provide details of structures they expect to have
made public before September 15, 2012. The last day for suggesting
proteins as CASP targets is July 16, 2012.
A target submission form is available at this website.
During the prediction season, targets are being posted daily at the
Target List page
and, additionally, automatically pushed to the registered prediction servers.
Targets in CASP10 will be split in two prediction tracks: those for 1) all groups
(long deadline) and 2) servers only (short deadline). Assignment of a target
to a particular track is made by the organizers and communicated to the predictors
through the Target List page. Priority for inclusion in the all groups modeling track
will be given to targets containing low homology domains.
Targets are planned to be released on business days only, around 9am PDT.
Tarballs for QA predictions will be released at noon, PDT.
Sequence and other relevant information about the targets will be
posted at the Target List web page.
Requests to the participating servers will be sent shortly after the
target release. We plan to release not more than 3
targets per day for servers and, usually, one target per day for regular
groups. All targets are assigned two expiration dates (one - for server predictors
and another - for regular groups). All predictions must be received
and accepted before noon, 12pm PDT on the corresponding expiration date.
We are planning to release 50-60 targets for evaluation in the all-group track
(long deadline) and as many targets as we can get in the server-only track.
Server groups are expected to submit their predictions for all targets.
Manual groups are expected to submit their predictions for long deadline targets (all_group)
in TS and RR categories, and for all targets (all_group + server_only) in
DR, FN and QA categories. Those manual groups that wish to submit
TS and RR predictions for server-only targets are welcome to do so, but these
predictions will not be officially evaluated in CASP. Note, that DR and FN
predictions from manual groups on server-only targets are due on the server
expiration date.
Predictions can be submitted through the Prediction Submission form available from
this web site or by the email provided at the format page. Please comply with the
instructions on CASP10 submission procedures and format.
Server predictions will be made publicly available shortly after the closing of prediction
window for a specific target. To enable the new testing procedure for QA methods, we will
be releasing server predictions in 3 stages: (1) up to 20 selected predictions spanning the whole
range of model accuracy will be released 2 days after the server TS deadline;
(2) best 150 server predictions (according to the ranking from the naive consensus QA method) -
4 days after the TS deadline; (3) all server predictions - 6 days after the server TS deadline
(see the QA format description at the format page).
If you are currently participating in CASP ROLL, you will have to submit your predictions
on the targets that were selected for both experiments only once - to CASP10. Our system
will automatically copy them to CASP ROLL.
As in previous CASPs, independent assessors will evaluate the predictions.
Assessors will be provided with the results of numerical prediction evaluations performed
at the Prediction Center, and will judge the results primarily on that basis.
They will be asked to focus particularly on the effectiveness of different methods.
Evaluation criteria will as far as possible be similar to those used in previous CASPs,
although the assessors are welcome to introduce additional measures.
There will be three assessors, focusing on the following areas of prediction:
-
Template based modeling -
Gaetano T. Montelione, Rutgers University, USA
-
Template free modeling -
BK Lee, NCI/NIH, Bethesda, USA
-
Refinement and physics-based prediction methods -
David Jones, University College London, UK
Other prediction categories (contacts, binding sites, disorder, quality assessment)
will be evaluated by the selected assessors or the organizers.
Click here
for the list of assessors in all CASPs held so far.
In accordance with CASP policy, assessors are not directly
involved in the organization of the experiment, nor can they take part
in the experiment as predictors. Predictors must not
contact assessors directly with queries, but rather these should be
sent to the
email
address.
All CASP predictions and results of numerical evaluation will be made available through
this web site shortly before the meeting.
The proceedings of the meeting will be published in a scientific journal (see publications of previous experiments).
All participants will also be required to describe their methods
in the abstracts (published locally at our web site) and encouraged to
discuss them on the
FORCASP forum.
These contributions will be discussed and scored
by other predictors, and this material will be taken into account in
choosing
some presentations at the meeting. Also, the predictors
presenting posters at the meeting should be prepared to give a
short
presentation at one of the main sessions as some talks will be
invited during the meeting based on the
discussion of poster sessions.
The meeting to discuss results of the experiment will be held
at the Hotel Serapo in
Gaeta (Italy)
on December 9-12, 2012 (starting at 6pm on the 9th and ending in the
afternoon of the 12th).
Program of the meeting will be available in mid-November, 2012.
The meeting cost is expected to be in the range of 450-550 EURO per person
(including registration fee, hotel acommodation and all meals for the time
of Conference).
Some financial assistance will be available for the most successful
predictors and students.
John Moult, CASP president; IBBR, University of Maryland, USA
Krzysztof Fidelis, University of California, Davis, USA
Andriy Kryshtafovych, University of California, Davis, USA
Torsten Schwede, University of Basel, Switzerland
Anna Tramontano, University of Rome, Italy
David Baker, University of Washington
Nick Grishin, University of Texas
David Jones, University College, London
Justin MacCallum, University of New York, Stony Brook
Michael Sternberg, Imperial College, London
|
