Submissions

The prediction method must be embodied in a set of algorithms and a software system. The software system may comprise a set of programs. The algorithms and programs must be well described and must be available to NCATS for verification.

The submitted model must produce an estimate of the probability of a chemical being active in an assay as well as an active/inactive call (see sample file). The performance of the model will be evaluated by the area under the receiver operating characteristic curve using the activity estimates produced by the model and the balanced accuracy.

NCATS will begin accepting final submissions for all subchallenges in August 2014. To qualify for scoring, final submissions must be received by 11:59 p.m. ET on November 14, 2014.

Submissions are tracked by team. Participants may join multiple teams, but each team must have a unique grouping of participants and name. Teams must submit a written description of the methodological approach and the source code used to produce the final submission by 11:59 p.m. ET on December 12, 2014. Written description and source code must conform to the following conditions:

• Methodological description must be provided as either Portable Document Format (PDF) or plain text. It should be sufficiently self-contained to convey the overall approach and highlight any important parameters (and/or hyperparameters) and references therein.
• All relevant source files must be available in plain text form. If the code has external dependencies (e.g., machine learning libraries), please specify the versions used with appropriate links. Usage of any external data not part of Tox21 for training must also be mentioned.
• Clearly identify the targeted subchallenges where appropriate.
• All files must be available as either tar,zip, tar.gz, or tar.bz archive format for submission. The total size of the archive should not exceed 5MB.

Predictions submitted without these two components will be scored, but are not eligible to win.

Winners for each subchallenge will be selected based on the accuracy of submitted predictions.