NNP Arena

a leaderboard of NNP benchmarks, originally on ariwagen.com

Molecular Energy Accuracy

Name
OMol25's eSEN Conserving Small	4.40	0.21	0.14	0.91	1.52
UMA Small (OMol task)	4.41	0.21	0.14	0.91	1.53
MACE-ANI-CC	9.28	0.32	0.23	1.04	3.11
B97-3c	10.16	0.30	0.35	2.32	3.46
AIMNet2 (ωB97M-D3, new)	14.66	0.54	0.39	2.35	4.96
Egret-1	16.88	0.31	0.20	1.60	5.49
Prescient's StrainRelief MACE	19.55	0.57	0.61	4.46	6.64
GFN2-xTB	18.65	0.72	0.73	14.60	6.82
Orb-v3 (Conservative Inf. OMat)	21.33	0.88	0.97	7.70	7.53
eSEN-OAM	22.16	0.84	0.69	7.05	7.67
MACE-MP-0b2(Large)-D3BJ	26.76	0.81	1.12	14.60	9.49
MACE-MP-0b2(Large)	31.09	1.08	1.15	14.59	10.95

Overall scores are a weighted average of individual benchmark results.

The "Overall" scores were calculated using a multi-step process. First, we assigned difficulty scores to each component benchmark by computing the ratio of MAEs between GFN2-xTB and B3LYP. We then weighted each benchmark result by multiplying it by two factors: its difficulty score and the number of systems under study. Finally, we computed a weighted average of these adjusted values to generate the "Overall" performance metric.

Benchmark	Weight
GMTKN55	0.31
Folmsbee	0.38
TorsionNet206	0.27
Wiggle150	0.04

The GMTKN55 WTMAD-2 shown here is filtered to only include exclusively neutral, singlet, and elemental-organic subsets.

Periodic Accuracy

Name
r²SCAN-3c	0.97	3.68	1.71
Egret-1	2.61	4.38	3.09
MACE-MP-0b2(Large)-D3BJ	3.47	3.03	3.35
UMA Small (OMC task)	3.22	4.94	3.69
GFN2-xTB	5.38	7.76	6.03
UMA Small (OMol task)	7.51	4.48	6.68
AIMNet2 (ωB97M-D3, new)	9.32	19.41	12.10
MACE-MP-0b2(Large)	9.28	22.39	12.89
eSEN-OAM	9.16	32.35	15.54
Orb-v3 (Conservative Inf. OMat)	26.79	12.30	22.80

Overall scores are a weighted average of individual benchmark results.

The "Overall" scores were calculated using a multi-step process. First, we assigned difficulty scores to each component benchmark by computing the ratio of MAEs between GFN2-xTB and r²SCAN-3c. We then weighted each benchmark result by multiplying it by two factors: its difficulty score and the number of systems under study. Finally, we computed a weighted average of these adjusted values to generate the "Overall" performance metric.

Benchmark	Weight
X23b Lattice Energies	0.72
X23b Cell Volumes	0.28

Molecular Speed

Name
Orb-v3 (Conservative Inf. OMat)	0.15	0.15	0.18
OMol25's eSEN Conserving Small	0.15	0.15	0.18
Prescient's StrainRelief MACE	0.23	0.23	0.25
MACE-ANI-CC	0.23	0.23	0.26
MACE-MP-0b2(Large)-D3BJ	0.23	0.23	0.26
UMA Small (OMol task)	0.22	0.22	0.26
MACE-MP-0b2(Large)	0.24	0.24	0.27
Egret-1	0.31	0.32	0.35
AIMNet2 (ωB97M-D3, new)	15.51	16.22	35.79
eSEN-OAM	0.23	0.27

Average speed of running energy and force calculations on molecular and periodic systems. Calculations run on Nvidia A10G GPUs through Modal.

Periodic Speed

Name
Orb-v3 (Conservative Inf. OMat)	0.15	0.15	0.18
MACE-MP-0b2(Large)-D3BJ	0.23	0.23	0.26
UMA Small (OMol task)	0.22	0.23	0.26
MACE-MP-0b2(Large)	0.23	0.23	0.27
Egret-1	0.31	0.32	0.36
AIMNet2 (ωB97M-D3, new)	20.22	24.90	66.55
eSEN-OAM	0.23	0.23