How to Use the NOMAD Encyclopedia

by Martin Kuban for exciting nitrogen

Purpose: In this tutorial, the NOMAD Encyclopedia will be introduced. It will be explained how you can search for materials in your browser. Furthermore, you can learn to use the web API to download data from the NOMAD Encyclopedia.

---

1. What is the NOMAD Encyclopedia?

The NOMAD Encyclopedia presents a materials-oriented view on the data of the NOMAD Archive and allows to effectively browse, explore, and visualize the data of the NOMAD Repository. NOMAD Encyclopedia can be accessed through the GUI (Graphical User Interface, a web application), or using a web API (Application Programming Interface).

In the following tutorial, you will learn to browse the NOMAD Encyclopedia using the GUI. As an advanced topic, you will also learn how to view and download data using the API.

---

2. The Encyclopedia GUI

The GUI is introduced with a few examples, showing the main functionalities.

To access the GUI, simply click here, or type encyclopedia.nomad-coe.eu into your browser. You will be forwarded to the GUI web page, which looks like this:

The text boxes are meant to guide new users trough the page. You can click on them to get the next hint and make them disappear. For this tutorial, you do not need to follow the instructions in the text boxes, but you are invited to read them.

---

2.1 The 'Structure' section

On the search page, you can create queries to search the Encyclopedia. In the bottom of the page, you see the periodic table of elements. You can click on the elements of the periodic table to select them for your search.

Now, select Iridium (Ir).

After that, the search bar will appear. It shows all the criteria that are currently selected for the search.

Below the search bar is a box labelled 'Exclusive search'. If you tick this box, you will search for compounds that contain only the selected elements.

Next, click on the 'Properties' button above the periodic table of elements. The first material we are searching for is bulk and has a cubic unit cell, so let's select those properties.

Do not forget to click on the 'Add to query' button. You can check your search bar, to see if the new search criteria have been selected.

Next, click on 'Method' in the left part of the page. Here, you can filter your search results by the method that was used for the calculation. Since exciting uses (L)APW+lo as basis functions for solving the Kohn-Sham equations, we select this basis set and click on 'Add to query'.

After selecting those criteria, your search bar should look like this:

Now, click on the 'Search' button.

Since there is only one material that fits our search criteria, you will immediately see the result:

On this page you see an overview of all the properties that have been calculated for this material. On the top left corner you see the structure viewer. Here you can inspect the conventional unit cell. In the structure viewer you can rotate and move the unit cell using the left and right mouse-button, respectively. Furthermore you can choose to see the unit cell vectors or the atomic bonds, you can reset the original view to undo all the rotations of the unit cell by clicking on the circular arrow. Additionally, you can download a virtual-reality (VR) file to experience the crystal structure in 3D using virtual reality hardware.

On the top right you see an overview of the electronic structure, in the bottom you see details about the methodology that was used for the calculations for this material. We will focus on those later. First, let's click on the orange + in the right corner of the 'Structure' box. This will enlarge the 'Structure' section.

You now see more detailed informations on the structure of your material. On the left, you see a larger version of the structure viewer. Below it, you find more detailed informations on the structure, i.e. the lattice (in this case cubic, which was a criterion for search), the crystallographic point group and which Wyckoff sites are occupied by which species.

On the right part of the page, you see the following:

The top left part of this section, indicated by a red box, is a list of all available calculations, sorted by exchange correlation functional and code used. In the Encyclopedia, individual calculations are grouped together, if they can be used for a certain purpose (e.g., an equation-of-states fit). In this case, several calculations are grouped into an 'eos' calculation, i.e., for all calculations in this group, the computational parameters are the same, but the lattice parameters are varied in order to minimize the total energy (similar to what you did in the Volume optimization for cubic systems).

The bottom left part, indicated by a green box, presents a summary of all entries that are selected (indicated by the ticked boxes) in the 'red' section. You can click on the symbols (a)) or on the functional names (b)) to display the statistics in a table or graphically, or select the results of calculations with a certain functional, respectively. In the table, you see for each property its value for the calculation with the minimal total energy, and the minimum and maximum values amongst all selected calculations in square brackets.

On the right side (blue box), you see details on the calculation that is currently selected (indicated by the orange arrow) in the red box. Since we have an equation-of-states calculation selected, we see the total energies of all individual calculations in the group in the figure a). You can use the calculation selector (b)) to show details on each individual calculation below.
To get further explanations on the shown quantities, activate the switch 'Additional information' in the top right corner of the page and hover the mouse over the property of your interest.

Next, we want to perform a new search. In the top left corner of the page, you see the navigation tree. Click on 'Search' to return to the search screen.

To clear the search bar, click on 'Clear all'.

---

2.2 The 'Electronic structure' section

Click on 'Element' to select new elements for the next search. From the periodic table, select titanium (Ti) and oxygen (O).

Now, we want to take a look at the electronic structure. To select these properties for the search, click on 'Properties' below the search bar. In the 'Properties' section, select 'Properties' on the left. In the grey box 'Results containing…', select 'Band structure' and click on 'Add to query'.

Below the search bar, select 'Exclusive search'. Your search bar should look like this:

Now click on 'Search'. Also in this case, we find only one result:

Now, click on the orange + in the right corner of the 'Electronic structure' section. You will see details on the electronic structure:

On the left side of this screen, you see again a list of calculations, like in the 'Structure' section. On the right side you see a box labelled 'Electronic structure'.

In the upper part of the screen, you see the band structure and the density of states that were obtained with this calculation. Again, you can move and zoom these plots using your mouse.
Please notice that slightly different electronic structures can be seen, if you select a different calculation in the list on the left. These changes origin in the different computational parameters that are used in each individual calculation.
You can identify calculations with band-structure and density-of-states data by the letter B and D next to the calculation ID, respectively.
If you have a band-structure calculation selected, you see the box labelled 'Brillouin zone' below the electronic-structure plots. In this interactive figure, you can see the Brillouin zone of your selected material. In the figure, you can rotate, translate and zoom the plot to explore it from all directions.
Additionally, the k-path that was used for the band-structure calculation is indicated in this figure. So you can see which regions of the Brillioun zone are shown in the band-structure calculation.

Let's perform a new search. Return to the search screen by selecting 'Search' in the navigation tree. Remember to click on 'Clear all' to start with a new query.

---

2.3 The 'Thermal properties' section

Now, navigate to the 'Properties' section and select 'Results containing…' thermal properties.

Your search bar should look like this:

Click on 'Search'. Since this time we have performed a rather 'unspecific' search, we have several results:

The materials shown here differ by their chemical composition, system type, space group, or/and structure type. Notice that we found 28 results in total, but on this page are only 10 of them displayed. You can click the 'next >' and '< prev' buttons in the top of the page, to navigate through the result pages. From this list we select Copper Chloride (CuCl), the last entry on the first page.

On the overview page of this material, you notice a new box in the bottom right of the page.

Again, click on the orange +, to inspect the details. You will see the following:

Here, you can explore the phonon dispersion relation, and specific heat and Helmholtz free energy as a function of the temperature in interactive figures. Also in this case, you can select different calculations from the 'Calculations' section on the left. You can identify calculations with thermal properties by the letter T next to the calculation ID.

Next, we want to see details on the methodology that was used for these calculations. In the navigation tree, select 'Methodology' from the drop-down menu:

You will see the following:

Each of these entries corresponds to a calculation for this material. You can inspect the calculation ID, type of calculation, the a full description of the used exchange-correlation functional, the code version, potential treatment and basis set. On the left side of the screen you can select which calculations you want to see on the right side of the screen. If you click on 'Filtering' in the top right corner of the screen, you will see the available filters, that allow you to see only the selected calculations.

---

As an advanced topic, you can also learn how to access the data of the Encyclopedia using the web API. Click the text below to show the explanation.

[How to use the Encyclopedia API]

3. The Encyclopedia API

In the next step, you will learn how to access the data of the NOMAD Encyclopedia using the API and the Python programming language.
Before we start downloading data, you have to learn a bit about the architecture of the NOMAD Encyclopedia: All input and output files that are uploaded to the NOMAD Repository are parsed and stored in the code-independent NOMAD Archive. The data of the Archive is then processed and grouped to materials, which are stored in the Encyclopedia database. In this database, each material has a 'material ID'. Using this ID, you can find all the properties that apply to the material independent of the calculation. Furthermore, each material has several associated calculations. These calculations are accessible using the 'calculation ID'. To get the properties that belong to a certain calculation, you need both the material ID and the calculation number.

---

3.1 Authentication

In order to use the Encyclopedia API, you need to get an authentication token. To get the token, please go to https://encyclopedia.nomad-coe.eu/api/v1.0/saml/ .

Here is the initial login window where you can log in to NOMAD, use a guest account or register for a new user account. For this tutorial you can just click on Guest Login:

The identity-provider (IdP) server will store some information about the user and here you should click on 'Accept':

After clicking 'Accept' the new token is generated and sent to your browser. If it does not show up in your browser, go to https://encyclopedia.nomad-coe.eu/api/v1.0/saml/user/ . You should see the following:

{
  "status": "Authenticated", 
  "token": {
    "data": "eyJpYXQiOjE1MzE3NTU1MTUsImV4cCI6MTUzMTg0MTkxNSwiYWxnIjoiSFMyNTYifQ.eyJpZCI6Im5vbWFkIn0.mX6rwSwt69ySKf3PAGOOy0ThaTgj3Y4IJJ1SlprAQi4", 
    "expires_in": 86400
  }, 
  "user": {
    "email": "[e-mail]", 
    "username": "nomad"
  }
}

Your token is the long string after the keyword "data".

The generated token will also have an expiration time embedded in it, it will expire after 24 hours. If you want to get large amounts of data from the API, you have to renew your token after this time.
After receiving you access token, you can start to make requests to the API.

Next, we will start with the exploration of the API. If you want to learn more about the API, you can find the full documentation at https://encyclopedia.nomad-coe.eu/doc/ . In this documentation you will find a full description of all endpoints of the API and how to access the data using these endpoints.

The Encyclopedia API URL which we will use in our requests is https://encyclopedia.nomad-coe.eu/api/v1.0/ .

---

3.2 Querying the API

This tutorial will use Python 3. Open the terminal and type:

$ ipython3

First, we have to import the module requests, which will be used to communicate with the NOMAD Encyclopedia API.

In the ipython command-line type the following command:

import requests

Now we can start defining some useful variables:

nomad_api_url ="https://encyclopedia.nomad-coe.eu/api/v1.0/materials"
api_access_token ="<add the API access token you got with the authentication here>"

In the case of the variable api_access_token, you have to add the actual authentication token between the double-quotes.
Now we can start searching for materials. This we do similar to the GUI: First we submit a search request to find the materials, then we can inspect the individual calculations. To submit your request, type in the ipython terminal:

r = requests.post(nomad_api_url, auth=(api_access_token, ''), json = {"search_by":{"element":"C", "pagination":"off"}, "has_dos":"true"})

The parameter "pagination":"off" is used to show all the results in one page. This can be used with up to 10000 results, afterwards you should use pages and perform a new request for each page. Furthermore, we specify our search to include only results with DOS-calculations ("has_dos":"true").

You can check if you request was successful by typing:

r.status_code

This status code is used in the Hypertext Transport Protocol. You can check the meaning of these status codes here, but for now you only have to check if the status code is $200$. This is the standard response for a successful request. If your status code is different, then you should check your request carefully and submit it again.
Now we can inspect the content of our request. To do so, we translate the response into the json-format by typing:

content = r.json()

And print it to the screen with:

print(content)

To get to know the structure of this data, type:

print(content.keys())

You see that this variable content consists of three parts:

• pages contains information about the pages of results that you got, similar to the results page on the GUI
• results contains all the materials on the current page, that have calculations matching our search query
• total_results contains the total number of materials that were found

You can print the total number of results with:

print(content['total_results'])

You can check this number of results with:

print(len(content['results']))

Let's print the formula and the space group for each material found:

print("formula","|","space_group")
for material in content['results']:
    print(material['formula'], "\t|", material['space_group'])

As mentioned above, each calculation in the Encyclopedia database is identified by a material ID and a calculation ID. You can find the material ID in the URL on the website, e.g. for diamond https://encyclopedia.nomad-coe.eu/gui/#/material/37749 you can read the material ID (37749) from the URL. In case of our search with the API, we can print the material ID from the results:

for material in content['results']:
    print(material['id'])

Let's start with an example, diamond. As seen above, its material ID is 37749. So let's take this entry from our results:

[diamond_material] = [x for x in content['results'] if x['id'] == 37749]

At least one of the calculations for diamond matches our search criteria ("has_dos":"true"). To find these calculations, we can inspect the content of the results:

print(diamond_material.keys())

print(diamond_material['calculations_list_matching_criteria'])

Let's choose one of these calculations (calculation ID: 72203) and extract the DOS. To access a calculation, we need a slightly different request. After the request was successful, we continue with the same steps as before.

diamond_calculation = requests.get(nomad_api_url+'/'+str(diamond_material['id'])+'/calculations'+'/72203', auth = (api_access_token,''))

diamond_data = diamond_calculation.json()

print(diamond_data.keys())

Behind each of those keys you can find a property of the material that was obtained with this specific calculation. You can access those properties using the keys, e.g. for the cell volume, type:

print(diamond_data['cell_volume'])

Let's save the volume to a variable for now:

volume = diamond_data['cell_volume']

Let's now visualize the DOS. To do so, we can either use the key 'diagram_data', or we can perform a new request and add '?property=dos' to the query:

diamond_dos = requests.get(nomad_api_url+'/'+str(diamond_material['id'])+'/calculations'+'/72203'+'?property=dos', auth = (api_access_token,''))

Let's inspect the keys of this response:

print(diamond_dos.json().keys())

Apparently, we have to use the key 'dos' to access the data.

print(diamond_dos.json()['dos'].keys())

finally gives access to the dos energies and values. Let's write them into variables:

energies = diamond_dos.json()['dos']['dos_energies']

The 'dos_values' also have the option of spin-polarized calculations, although the current calculation is not spin-polarized. Thus, we have to add $[0]$ to access the first (and only) entry for the DOS values.

dos = diamond_dos.json()['dos']['dos_values'][0]

In the Encyclopedia, all values are stored in SI units. A more common way to visualize the DOS is to view it per unit cell and convert the energies to eV. First, we define the electron charge as a variable:

electron_charge = 1.602176565e-19

Let's first rescale the energies:

energies = [x / electron_charge for x in energies]

To rescale the density of states, we also have to divide by the cell volume:

dos = [x / electron_charge / volume for x in dos]

Now we can use matplotlib (or any other plotting tool of your choice) to visualize the results:

import matplotlib.pyplot as plt

plt.plot(energies,dos)
plt.title('DOS of diamond')
plt.xlabel('Energy [eV]')
plt.ylabel(r'$DOS [\frac{states}{eV \cdot unit cell}$]')
plt.show()

You should see the following:

After using the API you should log out from NOMAD:

In your browser, type the following URL: https://encyclopedia.nomad-coe.eu/api/v1.0/saml/logout/ .You will get a page where you can confirm the logout:

After the logout is confirmed you are logged out from NOMAD services: