Tools for making OPTIMADE APIs from various formats of structural data (e.g. an archive of CIF files).

This repository contains the src/optimade-maker Python package and the corresponding CLI tool optimake, which together provide this functionality. Features include

• definition of a config file format (optimade.yaml) for annotating data archives to be used in the OPTIMADE ecosystem;
• conversion of the raw data into corresponding OPTIMADE types using pre-existing parsers (e.g., ASE for structures);
• conversion of the annotated data archive into the OPTIMADE JSON Lines file format (spec) that can be ingested into a database and used to serve a full OPTIMADE API.
• serving either an annotated data archive or a JSON Lines file as an OPTIMADE API (using the optimade-python-tools reference server implementation).

Install with

this will also make the optimake CLI utility available.

For a folder containing the data archive and the optimade.yaml file (such as in /examples), run

• optimake convert . to convert the entry into the JSONL format (see below).
• optimake serve . to start the OPTIMADE API (this also converts the entry, if needed);

For more detailed information, see optimake --help.

The sections below provide a high-level overview of the functionality, but a step-by-step notebook tutorial is available in examples/00_tutorial/tutorial.ipynb, demonstrating the full workflow from raw data to a running OPTIMADE API and example queries.

To run the tutorial locally, install optimade-maker[tutorial] and open the notebook with Jupyter.

To annotate your structural data for optimade-maker, the data archive needs to be accompanied by an optimade.yaml config file. The following is a simple example for a ZIP archive (structures.zip) of CIF files together with an optional property file (properties.csv):

See ./examples for a more complete set of supported formats and corresponding optimade.yaml config files.

optimade-maker will assign an id for each structure based on its full path in the archive, following a simple deterministic rule: from the set of all archive paths, the maximum common path prefix and postfix (including file extensions) are removed. E.g.

produces ["set1/101", "set2/102"].

The property files need to either refer to these ids or the full path in the archive to be associated with a structure. E.g. a possible property csv file could be

The toolkit supports a custom data pipeline (e.g. with an external MongoDB), by allowing to override any of the configuration passed to optimade-python-tools. See ./examples/override_config for details.

• OPTIMADE specification
• OPTIMADE specification: JSON Lines format

This project uses setuptools_scm, which reads the version from git tags. To release a new version:

This will trigger the Github Action that will create 1) a Github release; and 2) build and publish the package on pypi.

This project was funded by the NCCR MARVEL, a National Centre of Competence in Research, funded by the Swiss National Science Foundation (grant number 205602). We also acknowledge support by the Open Research Data Program of the Swiss ETH Board (project "API-03 IntER").