optimade 1.2.1 → 2.0.4

package/.jshintrc

@@ -0,0 +1 @@
+{"esversion": 6}

package/CHANGELOG.md

@@ -1,17 +1,31 @@
-# optimade changelog
+# Optimade changelog
+
+## 2.0.3
+
+- Bugfix release
+
+## 2.0.0
+
+- Add new `npm run prefetch` logic in prefetch.js to check provider avialability,
+  cache provider pagination limits in `provider.attributes.query_limits`,
+  introduce the sorted and structured providers.json (as source) and prefetched.json (as cache)
+- Add new arguments `getStructures(providerId, filter, page, limit)` for pagination
+  and pagination limits
+- Add new logic for catching errors like
+  `Error: messageFromProvider { response: { errors, meta } }`
 
 ## 1.2.1
 
-* Fix providers prefetching
+- Fix providers prefetching
 
 ## 1.2.0
 
-* Make batch data aggregation is optional to support per-provider progressive data receiving
+- Make batch data aggregation is optional to support per-provider progressive data receiving
 
 ## 1.1.5
 
-* Add http request timeout
+- Add http request timeout
 
 ## 1.0.0
 
-* First release
+- First release

package/README.md

@@ -1,26 +1,32 @@
 # Aggregating Optimade client for the online materials databases
 
-[![NPM version](https://img.shields.io/npm/v/optimade.svg?style=flat)](https://www.npmjs.com/package/optimade) [![NPM downloads](https://img.shields.io/npm/dm/optimade.svg?style=flat)](https://www.npmjs.com/package/optimade) [![GitHub issues](https://img.shields.io/github/issues/tilde-lab/optimade-client?style=flat)](https://github.com/tilde-lab/optimade-client/issues)
+[![NPM version](https://img.shields.io/npm/v/optimade.svg?style=flat)](https://www.npmjs.com/package/optimade)
+[![NPM downloads](https://img.shields.io/npm/dm/optimade.svg?style=flat)](https://www.npmjs.com/package/optimade)
+[![GitHub issues](https://img.shields.io/github/issues/tilde-lab/optimade-client?style=flat)](https://github.com/tilde-lab/optimade-client/issues)
 
 ## Features
 
 - discovers all the [official Optimade databases](https://providers.optimade.org) recursively
 - caches the discovered list (`prefetched.json`) for future and boosts the retrieval performance
 - queries them all, in the browser, at the server, everywhere
+- provides pagination, with the minimized number of pages
 
 ## Install
 
-```bash
+```sh
 npm i optimade --save
 ```
 
-```bash
+```sh
 yarn add optimade
 ```
 
-CDN: [UNPKG](https://unpkg.com/optimade/) | [jsDelivr](https://cdn.jsdelivr.net/npm/optimade/) (available as `window.optimade`)
+CDN: [UNPKG](https://unpkg.com/optimade/) |
+[jsDelivr](https://cdn.jsdelivr.net/npm/optimade/) (available as
+`window.optimade`)
 
-If you are **not** using es6 or CDN, add to your HTML just before closing the `body` tag:
+If you are **not** using ES6 or CDN, add to your HTML just before closing the
+`body` tag:
 
 ```html
 <script src="/path/to/optimade/dist/index.js"></script>
@@ -30,10 +36,19 @@ If you are **not** using es6 or CDN, add to your HTML just before closing the `b
 
 ## Usage
 
-```javascript
+The code is generally isomorphic, however one should additionally take care of
+downloading the cache or setting the CORS policy for the browsers. Concerning
+the CORS, the `Optimade` class constructor accepts the `corsProxyUrl` parameter,
+pointing to a running `cors-anywhere` proxy instance. This will be valid
+until all the Optimade providers are supplying the header
+`Access-Control-Allow-Origin $http_origin` in their responses. For the
+server-side environment this is not required.
 
+### Discovery and querying
+
+```ts
 const optimadeClient = new Optimade({
-    providersUrl: '/path/to/optimade/providers.json'
+  providersUrl: "https://providers.optimade.org/providers.json",
 });
 
 console.log(optimadeClient.providers); // { [id: string]: Provider }[]
@@ -42,13 +57,38 @@ const providersMap = await optimadeClient.getProviders(); // { [id: string]: Pro
 
 const providerIds = Object.keys(providersMap); // string[]
 
-const results = await optimadeClient.getStructuresAll(providerIds, YOUR_OPTIMADE_QUERY); // [Structures[], Provider][]
+const results = await optimadeClient.getStructuresAll(
+  providerIds,
+  YOUR_OPTIMADE_QUERY,
+); // [StructuresResponse[], Provider][]
 ```
 
-Importing depends on your environment. See also the `examples` folder. The `.html` examples are suited for the browser environment, the `.js` examples are suited for the server environment.
+Importing depends on your environment. See also the `examples` folder. The
+`.html` examples are suited for the browser environment, the `.js` examples are
+suited for the server environment.
+
+### Pagination
+
+```ts
+import prefetched from 'optimade/dist/prefetched.json';
+
+const optimadeClient = new Optimade({
+  providersUrl: "https://providers.optimade.org/providers.json",
+});
+
+optimadeClient.providers = prefetched.providers;
+optimadeClient.apis = prefetched.apis;
+
+const results = await optimadeClient.getStructuresAll(
+  providerIds,
+  YOUR_OPTIMADE_QUERY,
+  page: number,
+  limit: number
+); // [StructuresResponse[], Provider][]
+```
 
-The code is generally isomorphic, however one should additionally take care of downloading the cache or setting the CORS policy for the browsers. Concerning the CORS, the `Optimade` class constructor accepts the `corsProxyUrl` parameter, pointing to _e.g._  a running `cors-anywhere` proxy instance. This will be valid until all the Optimade providers are supplying the header `Access-Control-Allow-Origin $http_origin` in their responses. For the server-side environment this is not required.
+See also the `demo` folder.
 
 ## License
 
-MIT &copy; [PaulMaly](https://github.com/PaulMaly), Tilde Materials Informatics
+MIT &copy; [Pavel Malyshev](https://github.com/PaulMaly) and [Alexander Volkov](https://github.com/valexr), Tilde Materials Informatics

package/dist/index.d.ts

@@ -12,8 +12,8 @@ export declare class Optimade {
     });
     getProviders(api?: Types.Api): Promise<Types.ProvidersMap | null>;
     getApis(provider: Types.Provider | string, version?: string): Promise<Types.Api | null>;
-    getStructures(providerId: string, filter?: string): Promise<Types.Structure[] | null>;
-    getStructuresAll(providerIds: string[], filter?: string, batch?: boolean): Promise<Promise<Types.StructuresResult>[]> | Promise<Types.StructuresResult>[];
+    getStructures(providerId: string, filter: string, page: number, limit: number): Promise<Types.StructuresResponse[] | Types.ResponseError>;
+    getStructuresAll(providerIds: string[], filter: string, page: number, limit: number, batch?: boolean): Promise<Promise<Types.StructuresResult>[]> | Promise<Types.StructuresResult>[];
     private followLinks;
     private wrapUrl;
     private isDuplicatedReq;

package/dist/index.js

@@ -79,24 +79,44 @@
             const apis = await Optimade.getJSON(url);
             return Optimade.apiVersion(apis);
         }
-        async getStructures(providerId, filter = '') {
+        async getStructures(providerId, filter = '', page = 1, limit) {
             if (!this.apis[providerId])
                 return null;
             const apis = this.apis[providerId].filter(api => api.attributes.available_endpoints.includes('structures'));
-            const structures = await allSettled(apis.map((api) => {
-                const url = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter}` : '/structures');
-                return Optimade.getJSON(url);
+            const provider = this.providers[providerId];
+            const structures = await allSettled(apis.map(async (api) => {
+                if (page <= 0)
+                    page = 1;
+                const pageLimit = limit ? `&page_limit=${limit}` : '';
+                const pageNumber = page ? `&page_number=${page - 1}` : '';
+                const pageOffset = limit && page ? `&page_offset=${limit * (page - 1)}` : '';
+                const params = filter ? `${pageLimit + pageNumber + pageOffset}` : `?${pageLimit}`;
+                const url = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter + params}` : `/structures${params}`);
+                try {
+                    return await Optimade.getJSON(url, {}, { Origin: 'https://cors.optimade.science', 'X-Requested-With': 'XMLHttpRequest' });
+                }
+                catch (error) {
+                    return error;
+                }
             }));
             return structures.reduce((structures, structure) => {
-                return structure ? structures.concat(structure.data) : structures;
+                console.dir(`optimade-client-${providerId}:`, structure);
+                if (structure instanceof Error || Object.keys(structure).includes('errors')) {
+                    return structures.concat(structure);
+                }
+                else {
+                    structure.meta.pages = Math.ceil(structure.meta.data_returned / (limit || structure.data.length));
+                    structure.meta.limits = provider.attributes.query_limits || [10];
+                    return structures.concat(structure);
+                }
             }, []);
         }
-        getStructuresAll(providerIds, filter = '', batch = true) {
+        getStructuresAll(providerIds, filter = '', page = 1, limit, batch = true) {
             const results = providerIds.reduce((structures, providerId) => {
                 const provider = this.providers[providerId];
                 if (provider) {
                     structures.push(allSettled([
-                        this.getStructures(providerId, filter),
+                        this.getStructures(providerId, filter, page, limit),
                         Promise.resolve(provider)
                     ]));
                 }
@@ -125,9 +145,10 @@
             }
             const res = await fetchWithTimeout(url.toString(), { headers }, timeout);
             if (!res.ok) {
-                const err = new Error(res.statusText);
-                err.response = res;
-                throw err;
+                const err = await res.json();
+                const error = new Error(err.errors[0].detail);
+                error.response = err;
+                throw error;
             }
             if (res.status !== 204) {
                 return await res.json();

package/dist/index.mjs

@@ -75,24 +75,44 @@ class Optimade {
         const apis = await Optimade.getJSON(url);
         return Optimade.apiVersion(apis);
     }
-    async getStructures(providerId, filter = '') {
+    async getStructures(providerId, filter = '', page = 1, limit) {
         if (!this.apis[providerId])
             return null;
         const apis = this.apis[providerId].filter(api => api.attributes.available_endpoints.includes('structures'));
-        const structures = await allSettled(apis.map((api) => {
-            const url = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter}` : '/structures');
-            return Optimade.getJSON(url);
+        const provider = this.providers[providerId];
+        const structures = await allSettled(apis.map(async (api) => {
+            if (page <= 0)
+                page = 1;
+            const pageLimit = limit ? `&page_limit=${limit}` : '';
+            const pageNumber = page ? `&page_number=${page - 1}` : '';
+            const pageOffset = limit && page ? `&page_offset=${limit * (page - 1)}` : '';
+            const params = filter ? `${pageLimit + pageNumber + pageOffset}` : `?${pageLimit}`;
+            const url = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter + params}` : `/structures${params}`);
+            try {
+                return await Optimade.getJSON(url, {}, { Origin: 'https://cors.optimade.science', 'X-Requested-With': 'XMLHttpRequest' });
+            }
+            catch (error) {
+                return error;
+            }
         }));
         return structures.reduce((structures, structure) => {
-            return structure ? structures.concat(structure.data) : structures;
+            console.dir(`optimade-client-${providerId}:`, structure);
+            if (structure instanceof Error || Object.keys(structure).includes('errors')) {
+                return structures.concat(structure);
+            }
+            else {
+                structure.meta.pages = Math.ceil(structure.meta.data_returned / (limit || structure.data.length));
+                structure.meta.limits = provider.attributes.query_limits || [10];
+                return structures.concat(structure);
+            }
         }, []);
     }
-    getStructuresAll(providerIds, filter = '', batch = true) {
+    getStructuresAll(providerIds, filter = '', page = 1, limit, batch = true) {
         const results = providerIds.reduce((structures, providerId) => {
             const provider = this.providers[providerId];
             if (provider) {
                 structures.push(allSettled([
-                    this.getStructures(providerId, filter),
+                    this.getStructures(providerId, filter, page, limit),
                     Promise.resolve(provider)
                 ]));
             }
@@ -121,9 +141,10 @@ class Optimade {
         }
         const res = await fetchWithTimeout(url.toString(), { headers }, timeout);
         if (!res.ok) {
-            const err = new Error(res.statusText);
-            err.response = res;
-            throw err;
+            const err = await res.json();
+            const error = new Error(err.errors[0].detail);
+            error.response = err;
+            throw error;
         }
         if (res.status !== 204) {
             return await res.json();

package/dist/prefetched.json

@@ -1 +1 @@
-{"providers":{"aflow":{"id":"aflow","type":"links","attributes":{"name":"AFLOW","description":"The AFLOW OPTIMADE endpoint","base_url":"http://aflow.org/API/optimade/","homepage":"http://aflow.org","link_type":"child"}},"cod":{"id":"cod","type":"links","attributes":{"name":"Crystallography Open Database","description":"Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/cod/optimade","homepage":"https://www.crystallography.net/cod","link_type":"child"}},"sssp":{"id":"sssp","type":"links","attributes":{"name":"Standard solid-state pseudopotentials (SSSP)","description":"It contains data generated from the testing protocol of the standard solid-state pseudopotentials library.\n","base_url":"https://aiida.materialscloud.org/sssplibrary/optimade","homepage":"https://materialscloud.org/explore/sssplibrary","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"2dstructures":{"id":"2dstructures","type":"links","attributes":{"name":"2D Structures","description":"Two-dimensional (2D) materials from high-throughput computational exfoliation of experimentally known compounds.","base_url":"https://aiida.materialscloud.org/2dstructures/optimade","homepage":"https://materialscloud.org/explore/2dstructures","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"2dtopo":{"id":"2dtopo","type":"links","attributes":{"name":"2D topological insulators","description":"Quantum spin Hall insulator (QSHI) candidates through DFT band structure calculations of a database with 1825 compounds.","base_url":"https://aiida.materialscloud.org/2dtopo/optimade","homepage":"https://materialscloud.org/explore/2dtopo","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"tc-applicability":{"id":"tc-applicability","type":"links","attributes":{"name":"Applicability of tail-corrections in the molecular simulations of porous materials","description":"Tail-corrections in molecular simulations for adsorption of gasses in a diverse set of nanoporous crystalline materials (zeolites, Covalent Organic Framworks (COFs), and Metal Organic Frameworks (MOFs)).","base_url":"https://aiida.materialscloud.org/tc-applicability/optimade","homepage":"https://materialscloud.org/explore/tc-applicability","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"threedd":{"id":"threedd","type":"links","attributes":{"name":"Three-dimensional crystals database","description":"Curated set of relaxed three-dimensional crystal structures based on raw CIF data from the experimantal databases MPDS, COD, and ICSD.","base_url":"https://aiida.materialscloud.org/3dd/optimade","homepage":"https://materialscloud.org/explore/3dd","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"scdm":{"id":"scdm","type":"links","attributes":{"name":"Automated high-throughput Wannierisation","description":"Validation results of an automated protocol for generating maximally-localized Wannier functions in a high-throughput framework.","base_url":"https://aiida.materialscloud.org/autowannier/optimade","homepage":"https://materialscloud.org/explore/autowannier","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"curated-cofs":{"id":"curated-cofs","type":"links","attributes":{"name":"CURATED covalent organic frameworks database","description":"Database of experimentally reported Covalent-Organic Frameworks (COFs), provided with DFT-optimized geometry and DDEC partial charges for molecular simulations.","base_url":"https://aiida.materialscloud.org/curated-cofs/optimade","homepage":"https://materialscloud.org/discover/curated-cofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"optimade-sample":{"id":"optimade-sample","type":"links","attributes":{"name":"OPTIMADE Sample Database","description":"Database with example structures for OPTIMADE tests.","base_url":"https://aiida.materialscloud.org/optimade-sample/optimade","homepage":"https://materialscloud.org/explore/optimade-sample","link_type":"child","aggregate":"test","no_aggregate_reason":null}},"stoceriaitf":{"id":"stoceriaitf","type":"links","attributes":{"name":"SrTiO3-CeO2 interfaces","description":"Refining random structure searching results of SrTiO3-CeO2 interfaces, and exploring how it can affect ionic conduction. Candidate structures of the DFT validation calculations.","base_url":"https://aiida.materialscloud.org/stoceriaitf/optimade","homepage":"https://materialscloud.org/explore/stoceriaitf","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"pyrene-mofs":{"id":"pyrene-mofs","type":"links","attributes":{"name":"Pyrene MOFs","description":"Pyrene Metal Organic Frameworks (MOFs).","base_url":"https://aiida.materialscloud.org/pyrene-mofs/optimade","homepage":"https://materialscloud.org/discover/pyrene-mofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"li-ion-conductors":{"id":"li-ion-conductors","type":"links","attributes":{"name":"Solid-state Li-ion conductors.","description":"High-throughput computational screening study for solid-state Li-ion conductors.","base_url":"https://aiida.materialscloud.org/li-ion-conductors/optimade","homepage":"https://materialscloud.org/explore/li-ion-conductors","link_type":"child","aggregate":"staging","no_aggregate_reason":null}},"tin-antimony-sulfoiodide":{"id":"tin-antimony-sulfoiodide","type":"links","attributes":{"name":"Hidden spontaneous polarisation in the chalcohalide photovoltaic Sn2SbS2I3","description":"Structural, dynamic and electronic characterisation of a novel mixed-metal chalcohalide revealing hidden symmetry breaking and spontaneous polarisation.","base_url":"https://aiida.materialscloud.org/tin-antimony-sulfoiodide/optimade","homepage":"https://materialscloud.org/explore/tin-antimony-sulfoiodide","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"mp":{"id":"mp","type":"links","attributes":{"name":"The Materials Project","description":"The Materials Project OPTIMADE endpoint","base_url":"https://optimade.materialsproject.org","homepage":"https://www.materialsproject.org","link_type":"child"}},"mpds":{"id":"mpds","type":"links","attributes":{"name":"Materials Platform for Data Science","description":"A highly curated Pauling File dataset based on ~0.5M publications and backing up Springer Materials, ICDD PDF, ASM APD, MedeA, Pearson Crystal Data, AtomWork Advanced, etc.","base_url":"https://api.mpds.io","homepage":"https://mpds.io","link_type":"child"}},"nmd":{"id":"nmd","type":"links","attributes":{"name":"novel materials discovery (NOMAD)","description":"A FAIR data sharing platform for materials science data","base_url":"https://nomad-lab.eu/prod/rae/optimade/","homepage":"https://nomad-lab.eu","link_type":"child"}},"odbx":{"id":"odbx","type":"links","attributes":{"name":"odbx","description":"The Open Database of Xtals is run by the group of Dr Andrew Morris at the universities of Birmingham and Cambridge and is constructed using the matador library.","base_url":"https://optimade.odbx.science","homepage":"https://odbx.science","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"omdb_production":{"id":"omdb_production","type":"links","attributes":{"name":"Open Materials Database (omdb) production database","description":"This is the main production version of the Open Materials Database","base_url":"http://optimade.openmaterialsdb.se","homepage":"http://openmaterialsdb.se","link_type":"child"}},"oqmd":{"id":"oqmd","type":"links","attributes":{"name":"The OQMD","description":"The Open Quantum Materials Database endpoint","base_url":"http://oqmd.org/optimade/","homepage":"http://oqmd.org","link_type":"child"}},"jarvis":{"id":"jarvis","type":"links","attributes":{"name":"JARVIS-DFT","description":"JARVIS-DFT is a materials property repository focused on density functional theory (DFT) predictions of material properties, especially for crystalline materials.","base_url":"https://jarvis.nist.gov/optimade/jarvisdft","homepage":"https://jarvis.nist.gov","link_type":"child"}},"tcod":{"id":"tcod","type":"links","attributes":{"name":"Theoretical Crystallography Open Database","description":"Open-access collection of theoretically calculated or refined crystal structures of organic, inorganic, metal-organic compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/tcod/optimade","homepage":"https://www.crystallography.net/tcod","link_type":"child"}},"twodmatpedia":{"id":"twodmatpedia","type":"links","attributes":{"name":"2DMatpedia","description":"2DMatpedia, an open computational database of two-dimensional materials from top-down and bottom-up approaches","base_url":"http://optimade.2dmatpedia.org","homepage":"http://2dmatpedia.org","link_type":"child"}}},"apis":{"aflow":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"http://aflow.org/API/optimade/v0.9/","version":"0.9.0"},{"url":"http://aflow.org/API/optimade/v1.0/","version":"1.0.0"},{"url":"http://aflow.org/API/optimade/v1.1/","version":"1.1.0"}],"formats":["json"],"entry_types_by_format":{"json":["structures","calculations"]},"available_endpoints":["structures","calculations","info","links"]},"is_index":false}],"optimade-providers":[],"cod":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://www.crystallography.net/cod/optimade/v1.0.0/","version":"1.0.0"}],"available_endpoints":["structures","info"],"entry_types_by_format":{"json":["structures"]},"formats":["json"]},"id":"/","type":"info"}],"exmpl":[],"matcloud":[],"mcloud":[],"index":[],"sssp":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/sssplibrary/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"2dstructures":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/2dstructures/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"2dtopo":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/2dtopo/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"tc-applicability":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/tc-applicability/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"threedd":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/3dd/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"scdm":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/autowannier/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"curated-cofs":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/curated-cofs/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"optimade-sample":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/optimade-sample/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"stoceriaitf":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/stoceriaitf/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"pyrene-mofs":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/pyrene-mofs/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"li-ion-conductors":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/li-ion-conductors/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"tin-antimony-sulfoiodide":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/tin-antimony-sulfoiodide/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"mp":[{"id":"/","type":"info","attributes":{"api_version":"1.0.1","available_api_versions":[{"url":"http://optimade.materialsproject.org/v1","version":"1.0.1"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"mpds":[{"type":"info","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://api.mpds.io/v1","version":"1.0.0"}],"available_endpoints":["info","links","structures","extensions/properties","extensions/phase_diagrams","extensions/phases"],"formats":["json"],"entry_types_by_format":{"json":["info","links","structures","extensions/properties","extensions/phase_diagrams","extensions/phases"]}}}],"necro":[],"nmd":[{"id":"/","type":"info","attributes":{"api_version":"1.0.1","available_api_versions":[{"url":"https://nomad-lab.eu/prod/rae/optimade/v1","version":"1.0.1"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"odbx":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://optimade.odbx.science/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"omdb":[],"omdb_production":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"http://optimade.openmaterialsdb.se/v1","version":"1.0.0"},{"url":"http://optimade.openmaterialsdb.se/v1.0.0","version":"1.0.0"},{"url":"http://optimade.openmaterialsdb.se/v1.0","version":"1.0.0"}],"available_endpoints":["info","links","structures"],"entry_types_by_format":{"json":["structures"]},"formats":["json"],"is_index":false},"id":"/","type":"info"}],"oqmd":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"http://oqmd.org/optimade/v1/","version":"1.0.0"}],"formats":["json","xml","yaml"],"entry_types_by_format":{"json":["structures"],"xml":["structures"],"yaml":["structures"]},"available_endpoints":["structures","info","links","versions"],"is_index":false}}],"jarvis":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://jarvis.nist.gov/optimade/jarvisdft/v1","version":"1.0.0"}],"formats":["json","xml","yaml"],"entry_types_by_format":{"json":["structures"],"xml":["structures"],"yaml":["structures"]},"available_endpoints":["structures","info","links","versions"],"is_index":false}}],"tcod":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://www.crystallography.net/tcod/optimade/v1.0.0/","version":"1.0.0"}],"available_endpoints":["structures","info"],"entry_types_by_format":{"json":["structures"]},"formats":["json"]},"id":"/","type":"info"}],"twodmatpedia":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"http://optimade.2dmatpedia.org/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}]}}
+{"providers":{"2dstructures":{"id":"2dstructures","type":"links","attributes":{"name":"2D Structures","description":"Two-dimensional (2D) materials from high-throughput computational exfoliation of experimentally known compounds.","base_url":"https://aiida.materialscloud.org/2dstructures/optimade","homepage":"https://materialscloud.org/explore/2dstructures","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"2dtopo":{"id":"2dtopo","type":"links","attributes":{"name":"2D topological insulators","description":"Quantum spin Hall insulator (QSHI) candidates through DFT band structure calculations of a database with 1825 compounds.","base_url":"https://aiida.materialscloud.org/2dtopo/optimade","homepage":"https://materialscloud.org/explore/2dtopo","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"cod":{"id":"cod","type":"links","attributes":{"name":"Crystallography Open Database","description":"Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/cod/optimade","homepage":"https://www.crystallography.net/cod","link_type":"child","api_version":"1.0.0","query_limits":[10]}},"curated-cofs":{"id":"curated-cofs","type":"links","attributes":{"name":"CURATED covalent organic frameworks database","description":"Database of experimentally reported Covalent-Organic Frameworks (COFs), provided with DFT-optimized geometry and DDEC partial charges for molecular simulations.","base_url":"https://aiida.materialscloud.org/curated-cofs/optimade","homepage":"https://materialscloud.org/discover/curated-cofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"li-ion-conductors":{"id":"li-ion-conductors","type":"links","attributes":{"name":"Solid-state Li-ion conductors.","description":"High-throughput computational screening study for solid-state Li-ion conductors.","base_url":"https://aiida.materialscloud.org/li-ion-conductors/optimade","homepage":"https://materialscloud.org/explore/li-ion-conductors","link_type":"child","aggregate":"staging","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"mp":{"id":"mp","type":"links","attributes":{"name":"The Materials Project","description":"The Materials Project OPTIMADE endpoint","base_url":"https://optimade.materialsproject.org","homepage":"https://www.materialsproject.org","link_type":"child","api_version":"1.0.1","query_limits":[500]}},"mpds":{"id":"mpds","type":"links","attributes":{"name":"Materials Platform for Data Science","description":"A highly curated Pauling File dataset based on ~0.5M publications and backing up Springer Materials, ICDD PDF, ASM APD, MedeA, Pearson Crystal Data, AtomWork Advanced, etc.","base_url":"https://api.mpds.io","homepage":"https://mpds.io","link_type":"child","api_version":"1.0.0","query_limits":[5,10,100,750]}},"nmd":{"id":"nmd","type":"links","attributes":{"name":"novel materials discovery (NOMAD)","description":"A FAIR data sharing platform for materials science data","base_url":"https://nomad-lab.eu/prod/rae/optimade/","homepage":"https://nomad-lab.eu","link_type":"child","api_version":"1.0.1","query_limits":[500]}},"odbx":{"id":"odbx","type":"links","attributes":{"name":"odbx","description":"The Open Database of Xtals is run by the group of Dr Andrew Morris at the universities of Birmingham and Cambridge and is constructed using the matador library.","base_url":"https://optimade.odbx.science","homepage":"https://odbx.science","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[500]}},"optimade-sample":{"id":"optimade-sample","type":"links","attributes":{"name":"OPTIMADE Sample Database","description":"Database with example structures for OPTIMADE tests.","base_url":"https://aiida.materialscloud.org/optimade-sample/optimade","homepage":"https://materialscloud.org/explore/optimade-sample","link_type":"child","aggregate":"test","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"pyrene-mofs":{"id":"pyrene-mofs","type":"links","attributes":{"name":"Pyrene MOFs","description":"Pyrene Metal Organic Frameworks (MOFs).","base_url":"https://aiida.materialscloud.org/pyrene-mofs/optimade","homepage":"https://materialscloud.org/discover/pyrene-mofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"scdm":{"id":"scdm","type":"links","attributes":{"name":"Automated high-throughput Wannierisation","description":"Validation results of an automated protocol for generating maximally-localized Wannier functions in a high-throughput framework.","base_url":"https://aiida.materialscloud.org/autowannier/optimade","homepage":"https://materialscloud.org/explore/autowannier","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"sssp":{"id":"sssp","type":"links","attributes":{"name":"Standard solid-state pseudopotentials (SSSP)","description":"It contains data generated from the testing protocol of the standard solid-state pseudopotentials library.\n","base_url":"https://aiida.materialscloud.org/sssplibrary/optimade","homepage":"https://materialscloud.org/explore/sssplibrary","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"stoceriaitf":{"id":"stoceriaitf","type":"links","attributes":{"name":"SrTiO3-CeO2 interfaces","description":"Refining random structure searching results of SrTiO3-CeO2 interfaces, and exploring how it can affect ionic conduction. Candidate structures of the DFT validation calculations.","base_url":"https://aiida.materialscloud.org/stoceriaitf/optimade","homepage":"https://materialscloud.org/explore/stoceriaitf","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"tc-applicability":{"id":"tc-applicability","type":"links","attributes":{"name":"Applicability of tail-corrections in the molecular simulations of porous materials","description":"Tail-corrections in molecular simulations for adsorption of gasses in a diverse set of nanoporous crystalline materials (zeolites, Covalent Organic Framworks (COFs), and Metal Organic Frameworks (MOFs)).","base_url":"https://aiida.materialscloud.org/tc-applicability/optimade","homepage":"https://materialscloud.org/explore/tc-applicability","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"tcod":{"id":"tcod","type":"links","attributes":{"name":"Theoretical Crystallography Open Database","description":"Open-access collection of theoretically calculated or refined crystal structures of organic, inorganic, metal-organic compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/tcod/optimade","homepage":"https://www.crystallography.net/tcod","link_type":"child","api_version":"1.0.0","query_limits":[10]}},"threedd":{"id":"threedd","type":"links","attributes":{"name":"Three-dimensional crystals database","description":"Curated set of relaxed three-dimensional crystal structures based on raw CIF data from the experimantal databases MPDS, COD, and ICSD.","base_url":"https://aiida.materialscloud.org/3dd/optimade","homepage":"https://materialscloud.org/explore/3dd","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"tin-antimony-sulfoiodide":{"id":"tin-antimony-sulfoiodide","type":"links","attributes":{"name":"Hidden spontaneous polarisation in the chalcohalide photovoltaic Sn2SbS2I3","description":"Structural, dynamic and electronic characterisation of a novel mixed-metal chalcohalide revealing hidden symmetry breaking and spontaneous polarisation.","base_url":"https://aiida.materialscloud.org/tin-antimony-sulfoiodide/optimade","homepage":"https://materialscloud.org/explore/tin-antimony-sulfoiodide","link_type":"child","aggregate":"ok","no_aggregate_reason":null,"api_version":"1.1.0","query_limits":[100]}},"twodmatpedia":{"id":"twodmatpedia","type":"links","attributes":{"name":"2DMatpedia","description":"2DMatpedia, an open computational database of two-dimensional materials from top-down and bottom-up approaches","base_url":"http://optimade.2dmatpedia.org","homepage":"http://2dmatpedia.org","link_type":"child","api_version":"1.1.0","query_limits":[500]}}},"apis":{"2dstructures":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/2dstructures/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"2dtopo":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/2dtopo/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"aflow":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://aflow.org/API/optimade/v0.9/","version":"0.9.0"},{"url":"https://aflow.org/API/optimade/v1.0/","version":"1.0.0"},{"url":"https://aflow.org/API/optimade/v1.1/","version":"1.1.0"}],"formats":["json"],"entry_types_by_format":{"json":["structures","calculations"]},"available_endpoints":["structures","calculations","info","links"]},"is_index":false}],"cod":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://www.crystallography.net/cod/optimade/v1.0.0/","version":"1.0.0"}],"available_endpoints":["structures","info"],"entry_types_by_format":{"json":["structures"]},"formats":["json"]},"id":"/","type":"info"}],"curated-cofs":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/curated-cofs/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"jarvis":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://jarvis.nist.gov/optimade/jarvisdft/v1","version":"1.0.0"}],"formats":["json","xml","yaml"],"entry_types_by_format":{"json":["structures"],"xml":["structures"],"yaml":["structures"]},"available_endpoints":["structures","info","links","versions"],"is_index":false}}],"li-ion-conductors":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/li-ion-conductors/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"mp":[{"id":"/","type":"info","attributes":{"api_version":"1.0.1","available_api_versions":[{"url":"http://optimade.materialsproject.org/v1","version":"1.0.1"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"mpds":[{"type":"info","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://api.mpds.io/v1","version":"1.0.0"}],"available_endpoints":["structures","extensions/properties","extensions/phase_diagrams","extensions/phases"],"formats":["json"],"entry_types_by_format":{"json":["structures","extensions/properties","extensions/phase_diagrams","extensions/phases"]}}}],"nmd":[{"id":"/","type":"info","attributes":{"api_version":"1.0.1","available_api_versions":[{"url":"http://nomad-lab.eu/v1","version":"1.0.1"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"odbx":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://optimade.odbx.science/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}],"omdb_production":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"http://optimade.openmaterialsdb.se/v1","version":"1.0.0"},{"url":"http://optimade.openmaterialsdb.se/v1.0.0","version":"1.0.0"},{"url":"http://optimade.openmaterialsdb.se/v1.0","version":"1.0.0"}],"available_endpoints":["info","links","structures"],"entry_types_by_format":{"json":["structures"]},"formats":["json"],"is_index":false},"id":"/","type":"info"}],"optimade-sample":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/optimade-sample/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"oqmd":[{"type":"info","id":"/","attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"http://oqmd.org/optimade/v1/","version":"1.0.0"}],"formats":["json","xml","yaml"],"entry_types_by_format":{"json":["structures"],"xml":["structures"],"yaml":["structures"]},"available_endpoints":["structures","info","links","versions"],"is_index":false}}],"pyrene-mofs":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/pyrene-mofs/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"scdm":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/autowannier/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"sssp":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/sssplibrary/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"stoceriaitf":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/stoceriaitf/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"tc-applicability":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/tc-applicability/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"tcod":[{"attributes":{"api_version":"1.0.0","available_api_versions":[{"url":"https://www.crystallography.net/tcod/optimade/v1.0.0/","version":"1.0.0"}],"available_endpoints":["structures","info"],"entry_types_by_format":{"json":["structures"]},"formats":["json"]},"id":"/","type":"info"}],"threedd":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/3dd/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"tin-antimony-sulfoiodide":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"https://aiida.materialscloud.org/tin-antimony-sulfoiodide/optimade/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","extensions/docs","extensions/redoc","extensions/openapi.json","structures"],"entry_types_by_format":{"json":["structures"]},"is_index":false}}],"twodmatpedia":[{"id":"/","type":"info","attributes":{"api_version":"1.1.0","available_api_versions":[{"url":"http://optimade.2dmatpedia.org/v1","version":"1.1.0"}],"formats":["json"],"available_endpoints":["info","links","structures","references"],"entry_types_by_format":{"json":["structures","references"]},"is_index":false}}]}}

package/dist/providers.json

@@ -0,0 +1 @@
+{"2dstructures":{"id":"2dstructures","type":"links","attributes":{"name":"2D Structures","description":"Two-dimensional (2D) materials from high-throughput computational exfoliation of experimentally known compounds.","base_url":"https://aiida.materialscloud.org/2dstructures/optimade","homepage":"https://materialscloud.org/explore/2dstructures","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"2dtopo":{"id":"2dtopo","type":"links","attributes":{"name":"2D topological insulators","description":"Quantum spin Hall insulator (QSHI) candidates through DFT band structure calculations of a database with 1825 compounds.","base_url":"https://aiida.materialscloud.org/2dtopo/optimade","homepage":"https://materialscloud.org/explore/2dtopo","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"aflow":{"id":"aflow","type":"links","attributes":{"name":"AFLOW","description":"The AFLOW OPTIMADE endpoint","base_url":"http://aflow.org/API/optimade/","homepage":"http://aflow.org","link_type":"child"}},"cod":{"id":"cod","type":"links","attributes":{"name":"Crystallography Open Database","description":"Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/cod/optimade","homepage":"https://www.crystallography.net/cod","link_type":"child"}},"curated-cofs":{"id":"curated-cofs","type":"links","attributes":{"name":"CURATED covalent organic frameworks database","description":"Database of experimentally reported Covalent-Organic Frameworks (COFs), provided with DFT-optimized geometry and DDEC partial charges for molecular simulations.","base_url":"https://aiida.materialscloud.org/curated-cofs/optimade","homepage":"https://materialscloud.org/discover/curated-cofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"jarvis":{"id":"jarvis","type":"links","attributes":{"name":"JARVIS-DFT","description":"JARVIS-DFT is a materials property repository focused on density functional theory (DFT) predictions of material properties, especially for crystalline materials.","base_url":"https://jarvis.nist.gov/optimade/jarvisdft","homepage":"https://jarvis.nist.gov","link_type":"child"}},"li-ion-conductors":{"id":"li-ion-conductors","type":"links","attributes":{"name":"Solid-state Li-ion conductors.","description":"High-throughput computational screening study for solid-state Li-ion conductors.","base_url":"https://aiida.materialscloud.org/li-ion-conductors/optimade","homepage":"https://materialscloud.org/explore/li-ion-conductors","link_type":"child","aggregate":"staging","no_aggregate_reason":null}},"mp":{"id":"mp","type":"links","attributes":{"name":"The Materials Project","description":"The Materials Project OPTIMADE endpoint","base_url":"https://optimade.materialsproject.org","homepage":"https://www.materialsproject.org","link_type":"child"}},"mpds":{"id":"mpds","type":"links","attributes":{"name":"Materials Platform for Data Science","description":"A highly curated Pauling File dataset based on ~0.5M publications and backing up Springer Materials, ICDD PDF, ASM APD, MedeA, Pearson Crystal Data, AtomWork Advanced, etc.","base_url":"https://api.mpds.io","homepage":"https://mpds.io","link_type":"child"}},"nmd":{"id":"nmd","type":"links","attributes":{"name":"novel materials discovery (NOMAD)","description":"A FAIR data sharing platform for materials science data","base_url":"https://nomad-lab.eu/prod/rae/optimade/","homepage":"https://nomad-lab.eu","link_type":"child"}},"odbx":{"id":"odbx","type":"links","attributes":{"name":"odbx","description":"The Open Database of Xtals is run by the group of Dr Andrew Morris at the universities of Birmingham and Cambridge and is constructed using the matador library.","base_url":"https://optimade.odbx.science","homepage":"https://odbx.science","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"omdb_production":{"id":"omdb_production","type":"links","attributes":{"name":"Open Materials Database (omdb) production database","description":"This is the main production version of the Open Materials Database","base_url":"http://optimade.openmaterialsdb.se","homepage":"http://openmaterialsdb.se","link_type":"child"}},"optimade-sample":{"id":"optimade-sample","type":"links","attributes":{"name":"OPTIMADE Sample Database","description":"Database with example structures for OPTIMADE tests.","base_url":"https://aiida.materialscloud.org/optimade-sample/optimade","homepage":"https://materialscloud.org/explore/optimade-sample","link_type":"child","aggregate":"test","no_aggregate_reason":null}},"oqmd":{"id":"oqmd","type":"links","attributes":{"name":"The OQMD","description":"The Open Quantum Materials Database endpoint","base_url":"http://oqmd.org/optimade/","homepage":"http://oqmd.org","link_type":"child"}},"pyrene-mofs":{"id":"pyrene-mofs","type":"links","attributes":{"name":"Pyrene MOFs","description":"Pyrene Metal Organic Frameworks (MOFs).","base_url":"https://aiida.materialscloud.org/pyrene-mofs/optimade","homepage":"https://materialscloud.org/discover/pyrene-mofs","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"scdm":{"id":"scdm","type":"links","attributes":{"name":"Automated high-throughput Wannierisation","description":"Validation results of an automated protocol for generating maximally-localized Wannier functions in a high-throughput framework.","base_url":"https://aiida.materialscloud.org/autowannier/optimade","homepage":"https://materialscloud.org/explore/autowannier","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"sssp":{"id":"sssp","type":"links","attributes":{"name":"Standard solid-state pseudopotentials (SSSP)","description":"It contains data generated from the testing protocol of the standard solid-state pseudopotentials library.\n","base_url":"https://aiida.materialscloud.org/sssplibrary/optimade","homepage":"https://materialscloud.org/explore/sssplibrary","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"stoceriaitf":{"id":"stoceriaitf","type":"links","attributes":{"name":"SrTiO3-CeO2 interfaces","description":"Refining random structure searching results of SrTiO3-CeO2 interfaces, and exploring how it can affect ionic conduction. Candidate structures of the DFT validation calculations.","base_url":"https://aiida.materialscloud.org/stoceriaitf/optimade","homepage":"https://materialscloud.org/explore/stoceriaitf","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"tc-applicability":{"id":"tc-applicability","type":"links","attributes":{"name":"Applicability of tail-corrections in the molecular simulations of porous materials","description":"Tail-corrections in molecular simulations for adsorption of gasses in a diverse set of nanoporous crystalline materials (zeolites, Covalent Organic Framworks (COFs), and Metal Organic Frameworks (MOFs)).","base_url":"https://aiida.materialscloud.org/tc-applicability/optimade","homepage":"https://materialscloud.org/explore/tc-applicability","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"tcod":{"id":"tcod","type":"links","attributes":{"name":"Theoretical Crystallography Open Database","description":"Open-access collection of theoretically calculated or refined crystal structures of organic, inorganic, metal-organic compounds and minerals, excluding biopolymers","base_url":"https://www.crystallography.net/tcod/optimade","homepage":"https://www.crystallography.net/tcod","link_type":"child"}},"threedd":{"id":"threedd","type":"links","attributes":{"name":"Three-dimensional crystals database","description":"Curated set of relaxed three-dimensional crystal structures based on raw CIF data from the experimantal databases MPDS, COD, and ICSD.","base_url":"https://aiida.materialscloud.org/3dd/optimade","homepage":"https://materialscloud.org/explore/3dd","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"tin-antimony-sulfoiodide":{"id":"tin-antimony-sulfoiodide","type":"links","attributes":{"name":"Hidden spontaneous polarisation in the chalcohalide photovoltaic Sn2SbS2I3","description":"Structural, dynamic and electronic characterisation of a novel mixed-metal chalcohalide revealing hidden symmetry breaking and spontaneous polarisation.","base_url":"https://aiida.materialscloud.org/tin-antimony-sulfoiodide/optimade","homepage":"https://materialscloud.org/explore/tin-antimony-sulfoiodide","link_type":"child","aggregate":"ok","no_aggregate_reason":null}},"twodmatpedia":{"id":"twodmatpedia","type":"links","attributes":{"name":"2DMatpedia","description":"2DMatpedia, an open computational database of two-dimensional materials from top-down and bottom-up approaches","base_url":"http://optimade.2dmatpedia.org","homepage":"http://2dmatpedia.org","link_type":"child"}}}

package/dist/types.d.ts

@@ -19,6 +19,8 @@ export interface Meta {
         description: string;
         prefix: string;
     };
+    pages?: number;
+    limits?: number[];
 }
 export interface Links {
     base_url: string;
@@ -35,8 +37,8 @@ export interface ApiVer {
     [key: string]: string;
 }
 export interface Api {
-    type: string;
     id: string;
+    type: string;
     attributes: {
         api_version: string;
         available_api_versions: ApiVer | ApiVer[];
@@ -54,6 +56,8 @@ export interface Provider {
         base_url: string | null;
         homepage?: string | null;
         link_type?: string;
+        query_limits?: number[];
+        api_version?: string;
     };
 }
 export interface Structure {
@@ -78,7 +82,7 @@ export interface InfoResponse {
     [key: string]: any;
 }
 export interface StructuresResponse {
-    data: Structure[];
+    data?: Structure[];
     links?: Links;
     meta?: Meta;
 }
@@ -90,10 +94,20 @@ export interface LinksResponse {
 export interface ResponseError extends Error {
     response?: any;
 }
+export interface ErrorObject {
+    status: string;
+    title: string;
+    detail?: string;
+    length?: string;
+}
+export interface ErrorResponse {
+    errors: ErrorObject;
+    meta: any;
+}
 export declare type ProvidersMap = {
     [key: string]: Provider;
 };
 export declare type ApisMap = {
     [key: string]: Api[];
 };
-export declare type StructuresResult = [Promise<Structure>, Promise<Provider>][];
+export declare type StructuresResult = [Promise<StructuresResponse[]>, Promise<Provider>][];

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "optimade",
-  "version": "1.2.1",
+  "version": "2.0.4",
   "description": "Aggregating Optimade client for the online materials databases",
   "main": "dist/index.js",
   "module": "dist/index.mjs",

package/prefetch.js

@@ -1,5 +1,7 @@
 #!/usr/bin/env node
 
+/* jshint esversion: 6 */
+
 const fs = require('fs');
 const path = require('path');
 const { Optimade } = require('./dist/index');
@@ -9,19 +11,70 @@ const { AbortController } = require('node-abort-controller');
 global.AbortController = AbortController;
 
 const optimade = new Optimade({
-    providersUrl: 'https://providers.optimade.org/providers.json'
+	providersUrl: 'https://providers.optimade.org/providers.json'
 });
 
-optimade.getProviders().then(() => {
+optimade.getProviders().then(async () => {
+
+	const filteredApis = Object.entries(optimade.apis).filter(([k, v]) => v.length);
+	const apis = filteredApis.sort().reduce((acc, [k, v]) => {
+		return { ...acc, ...{ [k]: v } };
+	}, {});
+
+	const source = Object.keys(optimade.providers).sort().reduce(
+		(obj, key) => {
+			obj[key] = optimade.providers[key];
+			return obj;
+		}, {});
+
+	async function getQueryLimits(providers, max = 1000) {
 
-    const data = {
-        providers: optimade.providers,
-        apis: optimade.apis,
-    };
+		const fetchLimits = async (k, v) => {
+			const formula = `chemical_formula_anonymous="A2B"`;
+			const url = `${v.attributes.base_url}/v1/structures?filter=${formula}&page_limit=${max}`;
+			try {
+				const res = await fetch(url).then(res => res.json());
+				const api = res.meta && res.meta.api_version || apis[k][0].attributes.api_version;
+				//console.dir(res);
+				const detail = (e) => {
+					return e
+						? e.length
+							? e[0].detail
+							: e.detail
+						: '0';
+				};
+				const nums = detail(res.errors).match(/\d+/g).filter(n => +n < max).map(n => +n);
+				if (!nums.includes(0))
+					return {
+						[k]: { ...v, attributes: { ...v.attributes, api_version: api, ['query_limits']: nums } }
+					};
+			} catch (error) {
+				console.log(error);
+			}
+		};
 
-    fs.writeFile(path.join(__dirname, 'dist/prefetched.json'), JSON.stringify(data), (err) => {
-        if (err) throw err;
-        console.log('The cache file has been saved!');
-    });
+		providers = await Object.entries(providers).reduce(async (promise, [k, v]) => {
+			const provider = await fetchLimits(k, v);
+			const acc = await promise;
+			return { ...acc, ...provider };
+		}, Promise.resolve({}));
 
-});
+		//const log = { prefetched: Object.keys(providers).length, source: Object.keys(source).length };
+		//console.log(log);
+
+		return providers;
+	}
+
+	getQueryLimits(source).then(providers => {
+		const data = { providers, apis };
+		fs.writeFile(path.join(__dirname, 'dist/prefetched.json'), JSON.stringify(data), (err) => {
+			if (err) throw err;
+			console.log('The prefetched.json file has been saved!');
+		});
+		fs.writeFile(path.join(__dirname, 'dist/providers.json'), JSON.stringify(source), (err) => {
+			if (err) throw err;
+			console.log('The providers.json file has been saved!');
+		});
+	});
+
+});

package/src/index.ts

@@ -1,4 +1,3 @@
-
 import { allSettled, fetchWithTimeout } from './utils';
 
 import type * as Types from './types';
@@ -7,11 +6,11 @@ export { Types };
 export class Optimade {
     private providersUrl: string = '';
     private corsProxyUrl: string = '';
-    public providers: Types.ProvidersMap | null = null;
-    public apis: Types.ApisMap = {};
+    providers: Types.ProvidersMap | null = null;
+    apis: Types.ApisMap = {};
     private reqStack: string[] = [];
 
-    constructor({ providersUrl = '', corsProxyUrl = '' } : {providersUrl?: string; corsProxyUrl?: string} = {} ) {
+    constructor({ providersUrl = '', corsProxyUrl = '' }: { providersUrl?: string; corsProxyUrl?: string; } = {}) {
         this.corsProxyUrl = corsProxyUrl;
         this.providersUrl = this.wrapUrl(providersUrl);
     }
@@ -64,31 +63,48 @@ export class Optimade {
         return Optimade.apiVersion(apis);
     }
 
-    async getStructures(providerId: string, filter: string = ''): Promise<Types.Structure[] | null> {
+    async getStructures(providerId: string, filter: string = '', page: number = 1, limit: number): Promise<Types.StructuresResponse[] | Types.ResponseError> {
 
         if (!this.apis[providerId]) return null;
 
         const apis = this.apis[providerId].filter(api => api.attributes.available_endpoints.includes('structures'));
+        const provider = this.providers[providerId];
+
+        const structures: Types.StructuresResponse[] = await allSettled(apis.map(async (api: Types.Api) => {
+            if (page <= 0) page = 1;
+            const pageLimit = limit ? `&page_limit=${limit}` : '';
+            const pageNumber = page ? `&page_number=${page - 1}` : '';
+            const pageOffset = limit && page ? `&page_offset=${limit * (page - 1)}` : '';
+            const params = filter ? `${pageLimit + pageNumber + pageOffset}` : `?${pageLimit}`;
+            const url: string = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter + params}` : `/structures${params}`);
 
-        const structures: Types.StructuresResponse[] = await allSettled(apis.map((api: Types.Api) => {
-            const url: string = this.wrapUrl(Optimade.apiVersionUrl(api), filter ? `/structures?filter=${filter}` : '/structures');
-            // TODO pagination e.g. url += (filter ? '&' : '?') + 'page_limit=100'
-            return Optimade.getJSON(url);
+            try {
+                return await Optimade.getJSON(url, {}, { Origin: 'https://cors.optimade.science', 'X-Requested-With': 'XMLHttpRequest' });
+            } catch (error) {
+                return error;
+            }
         }));
 
-        //console.log('Ready ' + providerId);
-        return structures.reduce((structures: Types.Structure[], structure: Types.StructuresResponse | null) => {
-            return structure ? structures.concat(structure.data) : structures;
+        return structures.reduce((structures: any[], structure: Types.StructuresResponse | Types.ResponseError): Types.StructuresResponse[] => {
+            console.dir(`optimade-client-${providerId}:`, structure);
+
+            if (structure instanceof Error || Object.keys(structure).includes('errors')) {
+                return structures.concat(structure);
+            } else {
+                structure.meta.pages = Math.ceil(structure.meta.data_returned / (limit || structure.data.length));
+                structure.meta.limits = provider.attributes.query_limits || [10];
+                return structures.concat(structure);
+            }
         }, []);
     }
 
-    getStructuresAll(providerIds: string[], filter: string = '', batch: boolean = true): Promise<Promise<Types.StructuresResult>[]> | Promise<Types.StructuresResult>[] {
-        
+    getStructuresAll(providerIds: string[], filter: string = '', page: number = 1, limit: number, batch: boolean = true): Promise<Promise<Types.StructuresResult>[]> | Promise<Types.StructuresResult>[] {
+
         const results = providerIds.reduce((structures: Promise<any>[], providerId: string) => {
             const provider = this.providers[providerId];
             if (provider) {
                 structures.push(allSettled([
-                    this.getStructures(providerId, filter),
+                    this.getStructures(providerId, filter, page, limit),
                     Promise.resolve(provider)
                 ]));
             }
@@ -127,9 +143,10 @@ export class Optimade {
         const res = await fetchWithTimeout(url.toString(), { headers }, timeout);
 
         if (!res.ok) {
-            const err: Types.ResponseError = new Error(res.statusText);
-            err.response = res;
-            throw err;
+            const err: Types.ErrorResponse = await res.json();
+            const error: Types.ResponseError = new Error(err.errors[0].detail);
+            error.response = err;
+            throw error;
         }
 
         if (res.status !== 204) {

package/src/types.ts

@@ -18,7 +18,9 @@ export interface Meta {
         name: string;
         description: string;
         prefix: string;
-    };
+    },
+    pages?: number;
+    limits?: number[];
 }
 
 export interface Links {
@@ -38,8 +40,8 @@ export interface ApiVer {
 }
 
 export interface Api {
-    type: string;
     id: string;
+    type: string;
     attributes: {
         api_version: string;
         available_api_versions: ApiVer | ApiVer[];
@@ -58,6 +60,8 @@ export interface Provider {
         base_url: string | null;
         homepage?: string | null;
         link_type?: string;
+        query_limits?: number[];
+        api_version?: string;
     };
 }
 
@@ -87,7 +91,7 @@ export interface InfoResponse {
 }
 
 export interface StructuresResponse {
-    data: Structure[];
+    data?: Structure[];
     links?: Links;
     meta?: Meta;
 }
@@ -102,7 +106,18 @@ export interface ResponseError extends Error {
     response?: any;
 }
 
-export type ProvidersMap = { [key: string]: Provider }
-export type ApisMap = { [key: string]: Api[] }
+export interface ErrorObject {
+    status: string;
+    title: string;
+    detail?: string;
+    length?: string;
+}
+export interface ErrorResponse {
+    errors: ErrorObject;
+    meta: any;
+}
+
+export type ProvidersMap = { [key: string]: Provider; };
+export type ApisMap = { [key: string]: Api[]; };
 
-export type StructuresResult = [Promise<Structure>, Promise<Provider>][];
+export type StructuresResult = [Promise<StructuresResponse[]>, Promise<Provider>][];

package/src/utils.ts

@@ -14,7 +14,7 @@ export async function fetchWithTimeout(url, options = {}, timeout = 5000): Promi
 
     try {
         return await fetch(url, { ...options, signal });
-    } catch(err) {
+    } catch (err) {
         if (err.name === 'AbortError') {
             throw new Error('Request timed out');
         }