Rowan Benchmarks

Orb

Orb is a neural network potential (NNP) method from Mark Neumann and co-workers at Orbital Materials in 2024.

Orb is released under the Apache 2.0 license. Read the Orb preprint. See the Orb code.

Run Orb on Rowan

Use Orb to optimize structures, scan dihedrals, and much more with Rowan's simple GUI and cloud-based computing. We offer 500 free credits when you first sign up, and an additional 20 credits each week. Make a free account and get your first results within minutes!

Start computing →

Orb's GMTKN55 Results

GMTKN55 (download) is a high-accuracy molecular benchmark that measures basic properties, reaction energies, and noncovalent interactions. The GMTKN55 benchmark comprises 55 subsets. Each subset contains a number of relative energy measurements computed at the CCSD(T) level of theory. When viewing subset-level results, the mean absolute deviation/error (MAD/MAE) in kcal/mol is shown. Each subset is assigned a difficulty weight (see Section 4). When viewing category-level results, the weighted total mean absolute deviation (using the WTMAD-2 weights) is shown. See all GMTKN55 results.

NameIncomplete SubsetsBenchmarked By
ωB97M-D3BJ/def2-QZVP2.865.772.344.543.634.04link
B3LYP-D3BJ/def2-QZVP7.9910.164.115.654.826.18link
B97-3c11.9910.519.178.6211.8910.16link
GFN2-xTB20.0219.7713.6624.5811.4418.65link
Orb-v3 (Conservative Inf. OMat)26.0213.9612.7528.6717.6321.33
Orb-v3 (Direct Inf. OMat)24.0821.6616.6634.9220.9125.89
Orb-v3 (Conservative 20 OMat)21.0314.9115.2335.6232.7227.60
Orb-d3-v220.9714.2017.8346.6633.1631.92
Orb-v222.7033.0819.3542.7832.3833.57
Orb-v3 (Direct 20 OMat)26.4354.4316.5441.1041.6938.62

Orb's Folmsbee Results

The Folmsbee conformers benchmark (GitHub) is a high-accuracy molecular benchmark that evaluates the accuracy of predicting and ranking conformer relative energies. The benchmark contains 708 subsets of 10 conformers each, 632 of which have energies computed at the DLPNO-CCSD(T) level of theory. DLPNO-CCSD(T) calculations were not finished for 76 subsets, so every level of theory will show at least 76 incomplete subsets. Mean absolute error (MAE) and root mean square error (RMSE) are shown in kcal/mol. See all Folmsbee results.

NameIncomplete SubsetsBenchmarked By
ωB97X-D/def2-TZVP0.240.390.830.8580link
B97-3c0.300.530.800.8277link
GFN2-xTB0.721.320.570.5976link
Orb-v3 (Conservative Inf. OMat)0.881.710.510.50115
Orb-v3 (Conservative 20 OMat)1.052.110.490.44115
Orb-v3 (Direct Inf. OMat)1.152.060.440.41115
Orb-d3-v20.941.560.430.41115
Orb-v21.172.150.420.36115
Orb-v3 (Direct 20 OMat)1.302.330.400.36115

Orb's Speed Results

This is a benchmark that evaluates the average speed of running energy and force calculations on 10-, 100-, and 1000-atom molecular and periodic systems. All calculations were run on Nvidia A10G GPUs through Modal. Click items in legend to show/hide. See all periodic speed results.

Orb's TorsionNet206 Results

TorsionNet206 is a high-accuracy molecular benchmark that evaluates the accuracy of predicting and ranking of dihedral energy profiles. The benchmark contains 206 subsets of 24 conformers each computed at the CCSD(T)/def2-TZVP level of theory. Mean absolute error (MAE) and root mean square error (RMSE) are shown in kcal/mol. See all TorsionNet206 results.

NameIncomplete SubsetsBenchmarked By
ωB97M-D3BJ/vDZP0.150.180.990.98
B97-3c0.350.450.980.97
GFN2-xTB0.730.910.850.85
Orb-v3 (Conservative Inf. OMat)0.971.200.830.83
Orb-v3 (Conservative 20 OMat)1.161.420.800.82
Orb-v21.151.430.780.81
Orb-v3 (Direct Inf. OMat)1.822.310.750.76
Orb-d3-v21.431.730.740.76
Orb-v3 (Direct 20 OMat)1.602.010.720.74

Orb's Wiggle150 Results

Wiggle150 is a high-accuracy molecular benchmark that evaluates the accuracy of predicting and ranking of highly-strained conformers. The benchmark contains 150 highly-strained conformations of adenosine, benzylpenicillin, and efavirenz computed at the DLPNO–CCSD(T)/CBS level of theory. Mean absolute error (MAE) and root mean square error (RMSE) are shown in kcal/mol. See all Wiggle150 results.

NameBenchmarked By
Orb-v3 (Conservative Inf. OMat)7.728.82

Orb's X23b Results

X23b is a revised verison of X23, a periodic benchmark that evaluates the accuracy of reproducing experimental lattice energies and unit-cell volumes for 23 organic molecular crystals. Lattice energy (MAE) is shown in kcal/mol. Cell volume (MAPE, or mean average percent error) is shown in percentage. See all X23b results.

NameBenchmarked By
Orb-v3 (Conservative Inf. OMat)26.7912.30