Please use the tabs above to navigate to the individual categories for the results of the automatic evaluation, the submission abstracts and full details for each challenge and participant at the (+) links. At the end of this page there is a table legend.

The organising team would like to thank all participants for their contributions to the challenges this year and the many discussions on how to annotate and identify small molecules.

Category 1: Best Structure Identification on Natural Products

	F1 score	Mean rank	Median rank	Top	Top3	Top10	Misses	TopPos	TopNeg	Mean RRP	Median RRP	N
kai	469	235.02	4.0	11	19	28	4	9	2	0.965	0.999	45
kai112	350	266.00	3.0	12	14	16	17	9	3	0.985	1.000	45
CStacey	330	7.74	3.0	7	13	17	22	7	0	0.751	0.875	45
SamuelB2	319	12.12	4.0	6	12	21	16	4	2	0.929	0.982	45
SamuelB3	318	12.74	4.0	6	12	20	16	4	2	0.921	0.982	45
seeslab	237	18.29	13.0	4	10	15	12	3	1	0.926	0.957	45
SamuelB	229	15.43	9.0	4	8	17	16	2	2	0.904	0.931	45
Rakesh	192	1.89	1.0	6	8	9	36	4	2	0.998	1.000	45

The above table is also available as CSV download.

Category 2: Best Automatic Structural Identification - In Silico Fragmentation Only

	F1 score	Mean rank	Median rank	Top	Top3	Top10	Misses	TopPos	TopNeg	Mean RRP	Median RRP	N
kai_iso	2775	585.67	5.0	77	110	147	4	55	22	0.922	0.999	243
kai112	1373	513.93	9.0	38	55	74	102	28	10	0.937	0.999	243
yuanyuesimple	942	214.54	31.0	16	31	72	10	9	7	0.970	0.990	243
yuanyuesqrt	912	203.51	34.0	16	32	68	10	11	5	0.970	0.990	243
yuanyuelogsum	368	1165.47	260.0	7	14	27	10	5	2	0.831	0.941	243
Rakesh	109	939.83	333.0	0	0	5	28	0	0	0.852	0.906	243

The above table is also available as CSV download.

Category 3: Best Automatic Structural Identification - Full Information

	F1 score	Mean rank	Median rank	Top	Top3	Top10	Misses	TopPos	TopNeg	Mean RRP	Median RRP	N
tkind1	3707	4.33	2.0	91	148	193	34	52	39	0.644	0.750	238
tkind2	1704	15.20	6.0	29	63	107	72	15	14	0.755	0.846	240
TsugawaYamamoto	1559	19.99	7.0	28	56	104	64	16	12	0.841	0.900	243

The above table is also available as CSV download.

Category 4: Best Automatic Candidate Ranking

	F1 score	Mean rank	Median rank	Top	Top3	Top10	Misses	TopPos	TopNeg	Mean RRP	Median RRP	N
kai_iso	2306	658.28	5.0	66	91	119	0	46	20	0.913	0.999	198
kai112	2117	498.20	6.0	55	85	116	0	38	17	0.939	0.999	198
IOKRtransferAvgScore	1402	2740.56	201.0	36	59	73	0	20	16	0.681	0.955	198
IOKR_TanimotoGaussian	1212	3691.67	393.0	33	50	61	0	21	12	0.613	0.844	198
IOKRtransfer	1172	3733.52	396.0	31	48	60	0	19	12	0.609	0.832	198
yuanyuesimple	718	192.59	31.5	10	23	58	0	5	5	0.973	0.990	198
yuanyuesqrt	681	160.84	33.5	10	24	53	0	7	3	0.974	0.990	198
metfrag_plus	634	1750.18	142.0	6	21	51	0	2	4	0.811	0.966	198
metfrag	523	749.61	93.2	7	15	42	0	4	3	0.914	0.980	198
yuanyuelogsum	305	1248.99	253.2	5	12	22	0	4	1	0.833	0.956	198
Rakesh	301	607.01	135.5	0	2	19	2	0	0	0.904	0.962	198

The above table is also available as CSV download.

Table legend:

F1 score

The Formula 1 score awards points similar to the scheme in F1 racing for each challenge based on the rank of the correct solution. In the participant table, these are summed over all challenges. Please note that the F1 score is thus not neccessarily comparable across categories.

Mean/Median rank

Mean and median rank of the correct solution. For tied ranks with other candidates, the average rank of the ties is used.

Top, Top3, Top10

Number of challenges where the correct solution is ranked first, among the Top 3 and Top 10

Misses

Number of challenges where the correct solution is missing.

TopPos, TopNeg

Top1 ranked solutions in positive or negative ionization mode.

Mean/Median RRP

The relative ranking position, which is also incorporating the length of candidate list.

N

Number of submissions that have passed the evaluation scripts.