Protein–Ligand

protein–ligand binding and interaction benchmarks

FEP Benchmark Overview

Benchmark	N	Rowan FEP (TMD AM1-BCC)					Rowan FEP (TMD NAGL)					FEP+				OpenFE
Benchmark	N	RMSE	MAE	R²	ρ	min/lig	RMSE	MAE	R²	ρ	min/lig	RMSE	MAE	R²	ρ	RMSE	MAE	R²	ρ
JACS
TYK2	15	0.45	0.35	0.86	0.91	9.5	—	—	—	—	—	0.47	0.37	0.86	0.92	0.59	0.48	0.80	0.87
BACE	35	1.54	1.17	0.26	0.52	9.6	—	—	—	—	—	0.75	0.58	0.39	0.55	0.96	0.74	0.30	0.64
CDK2	15	1.13	0.82	0.45	0.68	12.3	—	—	—	—	—	0.90	0.72	0.42	0.63	0.84	0.70	0.60	0.82
JNK1	20	1.56	1.31	0.17	0.48	21.6	—	—	—	—	—	0.57	0.46	0.71	0.87	0.85	0.72	0.58	0.77
MCL1	41	1.33	0.97	0.50	0.74	12.5	—	—	—	—	—	0.86	0.69	0.49	0.67	1.17	0.91	0.49	0.71
p38	33	1.09	0.86	0.49	0.70	15.4	—	—	—	—	—	0.81	0.63	0.49	0.72	1.25	0.93	0.18	0.44
PTP1B	22	0.92	0.65	0.49	0.61	22.9	—	—	—	—	—	0.51	0.41	0.85	0.87	0.88	0.71	0.61	0.75
Thrombin	11	0.54	0.42	0.50	0.62	18.8	—	—	—	—	—	0.53	0.42	0.68	0.78	0.52	0.39	0.57	0.61
Average		1.07	0.82	0.46	0.66	15.3	—	—	—	—	—	0.68	0.53	0.61	0.75	0.88	0.70	0.52	0.70
Merck
CDK8	31	2.09	1.68	0.72	0.86	52.1	—	—	—	—	—	1.13	0.93	0.56	0.76	1.20	0.99	0.63	0.76
cMet (Neutral)	11	1.44	1.14	0.77	0.73	19.3	—	—	—	—	—	0.83	0.74	0.73	0.83	1.11	0.91	0.73	0.74
cMet (Positive)	11	1.40	1.16	0.60	0.78	30.8	—	—	—	—	—	0.62	0.51	0.78	0.93	1.56	1.25	0.06	0.26
Eg5 (Neutral)	8	1.15	1.06	0.63	0.52	21.4	—	—	—	—	—	0.64	0.55	0.54	0.78	1.01	0.84	0.59	0.65
Eg5 (Positive)	18	1.62	1.45	0.02	0.13	31.8	—	—	—	—	—	0.94	0.68	0.23	0.47	0.93	0.79	0.05	0.23
HIF2a	40	1.91	1.58	0.05	0.19	8.4	—	—	—	—	—	0.77	0.66	0.61	0.66	1.59	1.35	0.40	0.57
PFKFB3	39	1.18	0.94	0.27	0.52	12.1	—	—	—	—	—	0.99	0.77	0.58	0.76	1.11	0.90	0.41	0.63
SHP2	25	2.55	1.90	0.30	0.51	17.5	—	—	—	—	—	1.11	0.92	0.50	0.74	2.06	1.75	0.00	0.13
SYK (Negative)	5	2.56	2.13	0.62	-0.90	72.5	—	—	—	—	—	0.84	0.66	0.60	-0.94	2.40	1.68	0.81	-0.94
SYK (Neutral)	37	1.07	0.90	0.11	0.15	12.7	—	—	—	—	—	0.75	0.58	0.45	0.58	1.03	0.77	0.38	0.43
TNKS2 (Neutral)	20	0.79	0.61	0.41	0.62	8.6	—	—	—	—	—	0.69	0.58	0.63	0.63	0.78	0.60	0.46	0.56
TNKS2 (Positive)	5	0.89	0.81	0.02	-0.10	10.7	—	—	—	—	—	0.85	0.70	0.48	0.66	0.53	0.44	0.84	0.83
Average		1.55	1.28	0.38	0.33	24.8	—	—	—	—	—	0.85	0.69	0.56	0.57	1.28	1.02	0.45	0.40
Waterset
BRD4	7	2.22	1.90	0.25	-0.14	7.0	—	—	—	—	—	0.98	0.83	0.00	0.05	0.65	0.59	0.43	0.81
HSP90 (Kung)	10	1.59	1.04	0.71	0.66	8.2	—	—	—	—	—	1.14	0.92	0.69	0.64	3.31	2.55	0.47	0.77
Average		1.91	1.47	0.48	0.26	7.6	—	—	—	—	—	1.06	0.87	0.35	0.34	1.98	1.57	0.45	0.79
Misc
Galectin	25	0.91	0.69	0.57	0.66	14.2	—	—	—	—	—	0.42	0.36	0.76	0.84	0.88	0.72	0.53	0.76
HIV-1 Protease	12	1.20	0.89	0.02	0.34	238.0	1.27	1.12	0.04	-0.17	14.7	0.76	0.67	0.00	0.05	1.36	1.12	0.02	0.11
Average		1.05	0.79	0.29	0.50	126.1	1.27	1.12	0.04	-0.17	14.7	0.59	0.51	0.38	0.45	1.12	0.92	0.28	0.43
Overall Average		1.38	1.10	0.41	0.45	28.7	1.27	1.12	0.04	-0.17	14.7	0.79	0.64	0.54	0.60	1.19	0.95	0.46	0.54

Benchmark Details

Benchmark

Show Error Bars?

Method	R²	Spearman's ρ	MAE	RMSE
Rowan FEP (TMD AM1-BCC)	0.868	0.909	0.373	0.465
FEP+	0.862	0.918	0.371	0.475
OpenFE	0.799	0.871	0.476	0.587

This benchmark is from Ross et al. (2023). FEP+ results are from Ross et al. (2023). OpenFE results are from OpenFE Industry Benchmarks (2024).

Click a point to inspect a molecule