Compiling exciting

by Andris Gulans for exciting boron

Purpose: In this tutorial, you will learn more about the compilation of exciting.


1. General concerns

The build system of exciting is designed in a such way, that all environment- and compiler-dependent information is stored in build/make.inc. This file is generated once you run

$ make

for the first time and specify the basic compilation parameters as described in the first tutorial. This procedure assumes a ''typical'' system configuration. But, instead, you can tune make.inc yourself, and this is the prefered option. The default settings do not necessarily include the optimal choice of compiling flags. Furthermore, some of the provided templates sometimes imply linking with the netlib version of BLAS and LAPACK that comes along with exciting. Such a choice simplifies building binaries, but does not make them efficient in terms of performance.


2. Serial build

i) LAPACK library

Many algorithms in exciting make use of standard linear algebra operations such as matrix-matrix multiplication and diagonalization. Depending on the size of a studied system and a task, the linear algebra operations may become dominant. In such cases, it is important that BLAS and LAPACK are tuned for the optimal performance. Efficient implementations of these libraries are typically installed on supercomputing facilities, and you would need to know the location or to ask an administrator about it. Alternatively, you can install the libraries yourself. To do it, you have to choose which BLAS and LAPACK package to use. Three popular choices are Intel MKL, ACML, and OpenBLAS. A description, how to compile and/or install them, is provided with the packages or online.

In case you intend to use ACML, aplly the following changes to make.inc.

export USE_SYS_LAPACK=true
LIB_LPK = -L/path-to-lapack/ -lacml_mp

If you have installed OpenBLAS in /path-to-lapack/, you can link exciting with this library by adjusting the following lines in make.inc.

export USE_SYS_LAPACK=true
LIB_LPK = -L/path-to-lapack/ -lopenblas

Finally, if you prefer to link exciting with MKL and use the Intel fortran compiler it is sufficient to supply even a simpler flag.

export USE_SYS_LAPACK=true
LIB_LPK = -mkl=parallel

Note that, in these examples, exciting is linked with the library of your choice dynamically, and it is necessary to provide the access to it during the run time. In addition, its location has to be declared before running exciting. It may happen that the environment of your computing facility is already set up, and you do not have to worry about it. However, it may happen that when you try to run exciting, you get the error message containing

cannot open shared object file: No such file or directory

In this case, you have to run the following command before using exciting.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path-to-lapack/

The alternative option is to link the libraries statically. For example, if you use OpenBlas, specify the following in make.inc.
export USE_SYS_LAPACK=true
LIB_LPK = /path-to-lapack/libopenblas.a
ii) Threading support

Modern computers typically contain multi-core CPUs that can be involved in calculations simultaneously. It can be achieved by means of multithreading. exciting supports multithreading on two levels: (i) some parts of the code are explicitly parallelized using OpenMP and (ii) exciting uses standard BLAS and LAPACK routines, where multithreading is typically supported. In order to make sure that level (i) is enabled, make sure that compiler flags in make.inc contain -DUSEOMP and something similar to -openmp. The latter flag depends, on which compiler you use. For instance, it is -openmp, -fopenmp and -mp=allcores for ifort, gfortran, and pgf90, respectively. To enable level (ii), make simply sure that you link exciting with the version of the library that supports multithreading.

Once you have compiled excitingand are ready to run it, you can use the environment variable OMP_NUM_THREADS to control the number of threads during the run time. For example, the number of threads is limited to 4 if you run the following command.

export OMP_NUM_THREADS=4

In case you prefer not to use multithreading, set OMP_NUM_THREADS=1 or recompile exciting without -openmp -DUSEOMP.

Note that the script for generating make.inc offers to build the SMP version, which is synonimous with the multithreading support. This option is obsolete and will be removed from the script in future releases.

As the final remark, some architectures support hyperthreading that allocates two logical cores per each physical core, thus, doubling the maximum sensible number of threads. Such a technology is advantageous for memory-intensive applications. exciting, however, is computation-intensive. Therefore, hypertheading presents no gain in this case.

iii) Debug mode

The default compiling options are performance-minded, however, it is also possible to generate such a binary that performs a number of additional checks (division by zero, array-boundary violation, uninitialised variables, etc.) that help to spot bugs. This compiling mode is particularly useful for developers and occasionally for users who encounter code crashes.

The flags required for the debug mode depend on which compiler you use. For instance, the flags for ifort are shown below.

F90_OPTS=-g -check all -debug all -heap-arrays -traceback -implicitnone

These are the flags for gfortran.

F90_OPTS=-fcheck=all -fimplicit-none -fbacktrace -g
iv) Tests

Once exciting is compiled, it makes sense to check whether the binary works in the expected way. For this purpose, you can use a test suite supplied with the exciting source. It checks just the most basic functionality of exciting and does not require much time.

To test specifically the production version of exciting binary built possibly using aggressive optimization flags, edit make.inc, where you should set F90_DEBUGOPTS and F77_DEBUGOPTS to the same value as F90_OPTS and F77_OPTS, respectively.

To run the tests, execute make tests. If the test procedure does not terminate prematurely due to a crash of exciting or other reason, the results are summarized in test/report/index.html.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License