Open, build and test engine plugins¶
An engine plugin is a shared library that exports the rgpot_engine_* functions
of CppCore/rgpot/engine_c_abi.h.
A host opens it with dlopen and needs neither the library’s tensor runtime nor
its backend at link time.
This page shows how a host opens an engine, and how to build and test the engine
libraries.
Every function, its units and its error codes are in the
engine plugin ABI reference.
Open an engine¶
The header defines no search path: the host opens the path it was given. Check the major version before anything else, then the stamp when the engine exports it.
#include <dlfcn.h>
#include <stdio.h>
#include "rgpot/engine_c_abi.h"
int main(int argc, char **argv) {
if (argc < 2) {
fprintf(stderr, "usage: %s ENGINE_LIBRARY\n", argv[0]);
return 2;
}
void *h = dlopen(argv[1], RTLD_NOW | RTLD_LOCAL);
if (!h) {
fprintf(stderr, "%s\n", dlerror());
return 1;
}
int (*version)(void) = (int (*)(void))dlsym(h, "rgpot_engine_abi_version");
if (!version || version() != RGPOT_ENGINE_ABI_VERSION) {
fprintf(stderr, "not an rgpot engine of major %d\n", RGPOT_ENGINE_ABI_VERSION);
return 1;
}
int (*compatible)(const RgpotEngineAbiStamp *) =
(int (*)(const RgpotEngineAbiStamp *))dlsym(h, "rgpot_engine_abi_compatible");
const RgpotEngineAbiStamp mine = {RGPOT_ENGINE_ABI_VERSION, RGPOT_ENGINE_ABI_MINOR,
RGPOT_ENGINE_ABI_LAYOUT_REVISION};
if (compatible && !compatible(&mine)) {
fprintf(stderr, "engine is older than this host\n");
return 1;
}
const char *(*name)(void) = (const char *(*)(void))dlsym(h, "rgpot_engine_name");
printf("engine %s\n", name ? name() : "(unnamed)");
return 0;
}
A host without rgpot_engine_abi_compatible in the engine treats the engine as
minor 0, layout 1.
To create an instance, pass rgpot_engine_create the Cap’n Proto flat-array
message whose root is the library’s params struct (UmaParams, BuiltinParams,
…).
A message whose length is not a multiple of 8 bytes is refused with a reason in
errbuf.
Do not dlclose an engine: the tensor runtime keeps process-wide state.
libuma_engine.so is linked with -z nodelete, so a dlclose leaves it
mapped.
Destroy every handle before the process exits.
Build and test the engine libraries¶
The Meson build makes the engine libraries when -Dwith_rpc=true, because the
params messages need the Cap’n Proto schema.
libclassical_engine, libmopac_engine and libnwchem_engine need only that.
libxtb_engine also needs -Dwith_xtb=true,
libskala_engine -Dwith_skala=true, and
libmetatomic_engine -Dwith_metatomic=true.
libuma_engine needs -Dwith_metatomic=true and is not built with
-Dwith_python=true.
The classical, MOPAC, NWChem and Skala libraries are not built on Windows.
The umaenginebld pixi environment has the rpc, metatomic and xtb dependencies.
CI builds and tests every engine with the potctl command line tool (a cargo
binary in potctl/; cargo install --path potctl puts it on the PATH):
pixi run -e umaenginebld potctl ci ensure-torch-metatomic
pixi run -e umaenginebld potctl ci metatomic-test --rpc --build-dir bbdir-uma-engine
The first command installs a CPU torch and the metatensor and metatomic C++
packages into the environment when they are missing.
The second runs meson setup with -Dwith_tests=true, -Dwith_metatomic=true,
-Dwith_rpc=true, -Dwith_xtb=true and -Dwith_skala=true, then
meson compile and meson test.
Three Meson tests cover the engine table:
EngineAdapterTestruns the table over a stand-in backend (CppCore/tests/engine_stub.cc) without torch: argument checks, the transform, batching, error text, identity queries, the setting calls and eachreentrancypolicy.EngineTableTestopens each engine library the build made and compares the classical engine with the directly compiledLJPot; one classical handle serves eight threads. For the Metatomic engine it loads the models thatscripts/export_tiny_metatomic_model.pywrites into the build directory: they have no weights, and the energy moves by a known amount with the charge and multiplicity each one reads.UmaEngineTestopenslibuma_engineas a host does.
To run one of them in an existing build directory:
meson test -C bbdir-uma-engine --print-errorlogs EngineTableTest
The tests find the libraries through environment variables that Meson sets:
RGPOT_ENGINE_STUB and
RGPOT_ENGINE_STUB_MODEL for EngineAdapterTest, RGPOT_ENGINE_CLASSICAL, RGPOT_ENGINE_MOPAC, RGPOT_ENGINE_NWCHEM, RGPOT_ENGINE_XTB, RGPOT_ENGINE_SKALA, RGPOT_ENGINE_METATOMIC and RGPOT_ENGINE_EXPECT for EngineTableTest, and RGPOT_UMA_ENGINE
for UmaEngineTest.
With RGPOT_ENGINE_METATOMIC Meson also sets
MTA_CHARGE_SPIN_PT, MTA_CHARGE_PT, MTA_SPIN_PT, MTA_PLAIN_PT and MTA_UNDECLARED_PT
to the paths of those models.
UmaEngineTest needs model files for most of its cases, and skips them when the
variables are unset:
RGPOT_UMA_OMOL_PT2names a UMA package; with it the test compares the engine with the directly compiledUmaPot, checks the model digest against the file, checks the thread settings against libtorch, and runs four threads on one handle.RGPOT_UMA_OMOL_BAND_PT2names a band package for the batch case.UMA_FLOAT64_PT2names a small package that Meson exports withscripts/export_tiny_uma_contract.py; with it the test checks thatUmaParams.cacheDirplaces the unpack.RGPOT_UMA_REQUIRE_FIXTURES, when set and not empty, turns a missing fixture into a failure.
Two more tests cover the UMA unpack cache:
AotiUnpackTestchecks the cache without torch on small fake packages: the order of the cache roots, reuse, replacement of a broken unpack, concurrent unpacks from threads and processes, and that the system temporary directory gains no entry.UmaCacheDirTestloads theUMA_FLOAT64_PT2package withUmaConfig::cache_dir, withRGPOT_CACHE_DIR, and from several potentials at once.
Neither runs in parallel with other tests, because each checks that the system
temporary directory gains no entry.
Meson points RGPOT_CACHE_DIR of the UMA tests at uma-cache in the build
directory.