NNP Arena

a leaderboard of NNP benchmarks, originally on ariwagen.com

Molecular Energy Accuracy

Name
MACE-ANI-CC	9.28	0.32	0.23	1.04	3.11
B97-3c	10.16	0.30	0.35	2.32	3.46
AIMNet2 (ωB97M-D3, new)	14.66	0.54	0.39	2.35	4.96
Egret-1	16.88	0.31	0.20	1.60	5.49
Prescient's StrainRelief MACE	19.55	0.57	0.61	4.46	6.64
GFN2-xTB	18.65	0.72	0.73	14.60	6.82
Orb-v3 (Conservative Inf. OMat)	21.33	0.88	0.97	7.70	7.53
eSEN-OAM	22.16	0.84	0.69	7.05	7.67
MACE-MP-0b2(Large)-D3BJ	26.76	0.81	1.12	14.60	9.49
MACE-MP-0b2(Large)	31.09	1.08	1.15	14.59	10.95
Orb-d3-v2	31.92	0.94	1.43	9.27	11.03
Orb-v2	33.57	1.17	1.15	6.03	11.44
Orb-v3 (Direct 20 OMat)	38.62	1.30	1.60	9.01	13.29
SO3LR	69.98	2.65	2.24	10.36	23.80

Overall scores are a weighted average of individual benchmark results.

The "Overall" scores were calculated using a multi-step process. First, we assigned difficulty scores to each component benchmark by computing the ratio of MAEs between GFN2-xTB and B3LYP. We then weighted each benchmark result by multiplying it by two factors: its difficulty score and the number of systems under study. Finally, we computed a weighted average of these adjusted values to generate the "Overall" performance metric.

Benchmark	Weight
GMTKN55	0.31
Folmsbee	0.38
TorsionNet206	0.27
Wiggle150	0.04

Periodic Accuracy

Name
r²SCAN-3c	0.97	3.68	1.71
Egret-1	2.61	4.38	3.09
MACE-MP-0b2(Large)-D3BJ	3.47	3.03	3.35
GFN2-xTB	5.38	7.76	6.03
AIMNet2 (ωB97M-D3, new)	9.32	19.41	12.10
MACE-MP-0b2(Large)	9.28	22.39	12.89
eSEN-OAM	9.16	32.35	15.54
Orb-v3 (Conservative Inf. OMat)	26.79	12.30	22.80

Overall scores are a weighted average of individual benchmark results.

The "Overall" scores were calculated using a multi-step process. First, we assigned difficulty scores to each component benchmark by computing the ratio of MAEs between GFN2-xTB and r²SCAN-3c. We then weighted each benchmark result by multiplying it by two factors: its difficulty score and the number of systems under study. Finally, we computed a weighted average of these adjusted values to generate the "Overall" performance metric.

Benchmark	Weight
X23b Lattice Energies	0.72
X23b Cell Volumes	0.28

Molecular Speed

Average speed of running energy and force calculations on molecular and periodic systems. Calculations run on Nvidia A10G GPUs through Modal. Click items in legend to show/hide.

Periodic Speed