User Guide
User Guide
Contents
1 Introduction 1
1.1 People . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Contacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Guidelines for posting to the mailing list . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Terms of use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Installation 6
2.1 Download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Prerequisites for source compilation . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Building with CMake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Building with make . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4.2 Environment variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4.3 Supported architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4.4 Command-line options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4.5 configure for NVidia GPU’s . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.6 Manual configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.1 BLAS and LAPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5.2 FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.3 MPI libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.4 HDF5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.5 Other libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.6 In case of trouble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6 Libxc library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6.1 Linking in Quantum ESPRESSO . . . . . . . . . . . . . . . . . . . . . 15
2.6.2 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.6.3 Differences between Libxc and internal functionals . . . . . . . . . . . . . 16
2.6.4 Special cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.6.5 XC test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1
2.7 Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.8 Running tests and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.8.1 Test-suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.8.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.9 Installation tricks and problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.9.1 All architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.9.2 Linux PC’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.9.3 Linux PC clusters with MPI . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.9.4 Microsoft Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.9.5 Mac OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.9.6 Cray machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3 Parallelism 25
3.1 Understanding Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Running on parallel machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Parallelization levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4 Understanding parallel I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5 Tricks and problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1 Introduction
This guide gives a very general overview of Quantum ESPRESSO (opEn-Source Package for
Research in Electronic Structure, Simulation, and Optimization), version 7.5.0, and explains
how to build it from sources.
The Quantum ESPRESSO distribution contains the core packages PWscf (Plane-Wave
Self-Consistent Field) and CP (Car-Parrinello) for the calculation of electronic-structure prop-
erties within Density-Functional Theory (DFT), using a Plane-Wave (PW) basis set and pseu-
dopotentials. It also includes other packages for more specialized calculations:
• PWneb: energy barriers and reaction pathways through the Nudged Elastic Band (NEB)
method.
• GWL: electronic excitations within the GW approximation and with the Bethe-Salpeter
Equation
2
• QEHeat: energy current in insulators for thermal transport calculations.
• PWgui: a Graphical User Interface, producing input data files for PWscf and some PostProc
codes.
Several additional packages that exploit data produced by Quantum ESPRESSO or patch
some Quantum ESPRESSO routines can be downloaded and build together with Quantum
ESPRESSO, notably: make:
• GIPAW (Gauge-Independent Projector Augmented Waves): NMR chemical shifts and EPR
g-tensor.
For Quantum ESPRESSO with the self-consistent continuum solvation (SCCS) model, aka
“Environ”, see [Link]
Documentation on single packages can be found in the Doc/ directory of each package.
A detailed description of input data is available for most packages in files INPUT *.txt and
INPUT *.html.
The Quantum ESPRESSO codes work on many different types of Unix machines, in-
cluding parallel machines using both OpenMP and MPI (Message Passing Interface), as well
as machines running Mac OS X or MS-Windows. Since Feb.2021 NVidia GPU’s are supported
by the stable releases. AMD GPU’s are also supported but not yet in the main repository and
in stable releases.
Further documentation, beyond what is provided in this guide, can be found in:
• the archives of the mailing list: see section 1.2, “Contacts”, for more info;
3
This guide does not explain the basic Unix concepts (shell, execution path, directories etc.) and
utilities needed to run Quantum ESPRESSO; it does not explain either solid state physics
and its computational methods. If you want to learn the latter, you should first read a good
textbook, such as e.g. the book by Richard Martin: Electronic Structure: Basic Theory and
Practical Methods, Cambridge University Press (2004); or: Density functional theory: a prac-
tical introduction, D. S. Sholl, J. A. Steckel (Wiley, 2009); or Electronic Structure Calculations
for Solids and Molecules: Theory and Computational Methods, J. Kohanoff (Cambridge Uni-
versity Press, 2006). Then you should consult the documentation of the package you want to
use for more specific references.
All trademarks mentioned in this guide belong to their respective owners.
1.1 People
The maintenance and further development of the Quantum ESPRESSO distribution is pro-
moted by the Quantum ESPRESSO Foundation under the coordination of Paolo Giannozzi
(Univ. Udine and IOM-CNR, Italy) and Pietro Delugas (SISSA Trieste) with a strong support
from the MaX - Materials design at the Exascale EU Centre of Excellence and from the CINECA
computing centre. The NVidia GPU porting owes much to Pietro Bonfà (Univ. Modena), Ivan
Carnimeo (SISSA Trieste), Fabrizio Ferrari Ruffino (IOM-CNR).
Contributors to Quantum ESPRESSO, beyond the authors of the papers mentioned in
Sec.1.4, include:
• Fabio Affinito and Sergio Orlandini (CINECA) for ELPA support, for contributions to
the FFT library, and for various parallelization improvements;
• Alexandre Tkatchenko’s group, in particular Szabolcs Goger (U. Luxembourg), and Robert
DiStasio’s group, in particular Hsin-Yu Ko (Cornell), for Many-Body Dispersion (MBD)
correction;
• Federico Ficarelli and Daniele Cesarini (CINECA), with help from Ye Luo (Argonne) and
Sebastian Gsänger, for CMake support;
• Sebastiano Caravati for direct support of GTH pseudopotentials in analytical form, San-
tana Saha and Stefan Goedecker (Basel U.) for improved UPF converter of newer GTH
pseudopotentials;
• Axel Kohlmeyer for libraries and utilities to call Quantum ESPRESSO from external
codes (see the COUPLE sub-directory), made the parallelization more modular and usable
by external codes;
4
• Yves Ferro (Univ. Provence) for SOGGA and M06L functionals;
• Daniel Forrer (Padua Univ.) and Michele Pavone (Naples Univ. Federico II) for disper-
sions interaction in the framework of DFT-D;
• Filippo Spiga (University of Cambridge, now at NVidia) for mixed MPI-OpenMP paral-
lelization and for the first GPU-enabled version;
• Costas Bekas and Alessandro Curioni (IBM Zurich) for the initial BlueGene porting.
Åke Sandgren, Audrius Alkauskas, Alain Allouche, Francesco Antoniella, Uli As-
chauer, Francesca Baletto, Gerardo Ballabio, Mauro Boero, Scott Brozell, Claudia
Bungaro, Paolo Cazzato, Gabriele Cipriani, Jiayu Dai, Stefano Dal Forno, Cesar Da
Silva, Alberto Debernardi, Gernot Deinzer, Alin Marin Elena, Francesco Filipponi,
Prasenjit Ghosh, Marco Govoni, Thomas Gruber, Martin Hilgeman, Yosuke Kanai,
Konstantin Kudin, Nicolas Lacorne, Hyungjun Lee, Stephane Lefranc, Sergey Lisenkov,
Kurt Maeder, Andrea Marini, Giuseppe Mattioli, Nicolas Mounet, William Parker,
Pasquale Pavone, Mickael Profeta, Chung-Yuan Ren, Kurt Stokbro, David Strubbe,
Sylvie Stucki, Paul Tangney, Pascal Thibaudeau, Davide Tiana, Antonio Tilocca,
Jaro Tobik, Malgorzata Wierzbowska, Vittorio Zecca, Silviu Zilberman, Federico
Zipoli,
1.2 Contacts
The main entry point for Quantum ESPRESSO users is the web site:
[Link]
There you find the stable releases for download, general information and documentation.
The recommended place where to ask questions about installation and usage of Quantum
ESPRESSO, and to report problems, is the mailing list users@[Link].
Here you can obtain help from the developers and from knowledgeable users. You have to be
subscribed (see the “Contacts” section of the web site) in order to post to the users’ list. Please
check your spam folder if you do not get a confirmation message when subscribing.
Please read the guidelines for posting, section 1.3! PLEASE NOTE: only messages that
appear to come from the registered user’s e-mail address, in its exact form, will be accepted.
1
no longer maintained since some time: you may find the names of more recent contributors in merge requests
and issues on GitLab, [Link]
5
In case of trouble, carefully check that your return e-mail is the correct one (i.e. the one you
used to subscribe).
The main entry point for developers is the GitLab web site: [Link]
If you need to contact the developers for specific questions about coding, proposals, offers of
help, etc., you may either post an “Issue” to GitLab, or send a message to the developers’
mailing list developers@[Link]. Please do not post general questions
there: they will be ignored.
• Before posting, please: browse or search the archives – links are available in the “Contacts”
section of the web site. Most questions are asked over and over again. Also: make an
attempt to search the available documentation, notably the FAQs and the User Guide(s).
The answer to most questions is already there.
• Reply to both the mailing list and the author or the post, using “Reply to all”.
• Be short: no need to send 128 copies of the same error message just because this is what
came out of your 128-processor run. No need to send the entire compilation log for a
single error appearing at the end.
• Do not post large attachments: post a link to a place where the attachment(s) can be
downloaded from, such as e.g. DropBox, GoogleDocs, or one of the various web temporary
storage spaces.
• Avoid excessive or irrelevant quoting of previous messages. Your message must be imme-
diately visible and easily readable, not hidden into a sea of quoted text.
• Remember that even experts cannot guess where a problem lies in the absence of sufficient
information. One piece of information that must always be provided is the version number
of Quantum ESPRESSO.
• Remember that the mailing list is a voluntary endeavor: nobody is entitled to an answer,
even less to an immediate answer.
• Finally, please note that the mailing list is not a replacement for your own work, nor is
it a replacement for your thesis director’s work.
6
1.4 Terms of use
Quantum ESPRESSO is free software, released under the GNU General Public License.
See [Link] or the file License in the
distribution).
We shall greatly appreciate if scientific work done using the Quantum ESPRESSO dis-
tribution will contain an acknowledgment to the following references:
and
Users of the GPU-enabled version should also cite the following paper:
Note the form Quantum ESPRESSO for textual citations of the code. Please also see
package-specific documentation for further recommended citations. Pseudopotentials should
be cited as (for instance)
2 Installation
2.1 Download
Quantum ESPRESSO is distributed in source form, but selected binary packages, virtual
machines, dockers, may also be available. Stable and development releases of the Quantum
ESPRESSO source package (current version is 7.5.0), as well as available binary packages, can
be downloaded from the links listed in the “Download” section of [Link].
7
The Quantum Mobile virtual machine for Windows/Mac/Linux/Solaris provides a complete
Ubuntu Linux environment, containing Quantum ESPRESSO and much more. Link and
description in [Link]
For source compilation, uncompress and unpack compressed archives in the typical .[Link]
format using the command:
tar zxvf [Link]
(a hyphen before ”zxvf” is optional) where X.Y.Z stands for the version number.
A few additional packages that are not included in the base distribution will be downloaded
on demand at compile time, using either make or CMake (see Sec.2.7). Note however that this
will work only if the computer you are installing on is directly connected to the internet and
has either wget or curl installed and working. If you run into trouble, manually download each
required package into subdirectory archive/, not unpacking or uncompressing it: command
make will take care of this during installation.
The Quantum ESPRESSO distribution contains several directories. Some of them are
common to all packages:
Modules/ Fortran modules and utilities used by all programs
upflib/ pseudopotential-related code, plus conversion tools
include/ files *.h included by fortran and C source files
FFTXlib/ FFT libraries
LAXlib/ Linear Algebra (parallel) libraries
KS Solvers/ Iterative diagonalization routines
UtilXlib/ Miscellaneous timing, error handling, MPI utilites
XClib/ Exchange-correlation functionals (excepted van der Waals)
MBD/ Routines for many-body dispersions
dft-d3/ Routines for DFT-D3 disesive corrections
LR Modules/ Fortran modules and utilities used by linear-response codes
install/ installation scripts and utilities
pseudo/ pseudopotential files used by examples
Doc/ general documentation
external/ external libraries automatically downloaded
test-suite/ automated tests
while others are specific to a single package:
PW/ PWscf package
EPW/ EPW package
NEB/ PWneb package
PP/ PostProc package
PHonon/ PHonon package
PWCOND/ PWcond package
CPV/ CP package
atomic/ atomic package
GUI/ PWGui package
HP/ HP package
QEHeat/ QEHeat package
KCW/ KCW package
Finally, directory COUPLE/ contains code and documentation that is useful to call Quantum
ESPRESSO programs from external codes.
8
2.2 Prerequisites for source compilation
First of all, you need a minimal Unix environment: a command shell like bash or sh, utilities
make, awk, sed. Note that the scripts contained in the distribution assume that the local
language is set to the standard, i.e. ”C”; other settings may break them. Use export LC ALL=C
(sh/bash) or setenv LC ALL C (csh/tcsh) to prevent any problem when running those scripts.
If you are not compiling a stable releasei from tarballs, you will also need git v.2.13 or later
to get the external libraries. You will need either CMake v.3.20 or later, or the configure
command from autoconf v. 2.64 or later.
Second, you need a Fortran compiler compliant with F2008 standard and any half-decent
C compiler. For parallel execution, a parallel MPI-aware Fortran compiler and MPI libraries
implementing v.3 of the standard (notably, non-blocking broadcast and gather operations) are
required. For massively parallel machines, or for simple multicore parallelization, an OpenMP-
aware Fortran compiler and libraries are also required. To compile for NVidia GPUs you need
the NVidia HPC SDK (software development kit) v.21.7 or later, freely available for download.
As a rule, Quantum ESPRESSO tries to keep compatibility with older compilers, avoiding
nonstandard extensions and newer features that are not widespread or stabilized. If however
your compiler is older than a few (∼ 5) years, it is likely that something will not work. The
same applies to mathematical and MPI libraries.
Big computing centers typically provide a Fortran compiler complete with all needed li-
braries. Workstations or “commodity” machines using PC hardware, may or may not have
the needed software. If not, you may use the open-source gfortran compiler from the gcc dis-
tribution, and possibly open-source MPI libraries and run-time software. You may also get a
commercial compiler, some of which are available free of charge under some conditions (e.g.
academic or personal usage, no support) and may provide MPI libraries and run-time software
as well.
cd qe-X.Y.Z/
9
./configure
make all
This will (try to) produce parallel (MPI) executable if a proper parallel environment is detected,
serial executables otherwise. For OpenMP executables, specify ./configure --enable-openmp.
For GPUs, see GPU-specific instructions. Symlinks to executable programs appear in the bin/
subdirectory.
configure generates the following files:
[Link] compilation rules and flags (used by Makefile)
install/[Link] a report of the configuration run (not needed for compilation)
install/[Link] detailed log of the configuration run (useful for debugging)
include/configure.h optional: info on compilation flags (to enable it, uncomment
#define __HAVE_CONFIG_INFO in Modules/environment.f90)
In addition, configure generates (since v.7) files [Link], containing dependencies upon
modules, in the various subdirectories. If you add/remove/move/rename modules, or change
the list of objects in any Makefile, type make depend, or run ./install/[Link], to
update files [Link].
It is convenient to use ”parallel make” to speed up compilation: make -jN compiles in
parallel on N processors. Note that if you interrupt make while unpacking and compiling an
external library, you may run into trouble the next time you type make. If so, run make
veryclean, or even make distclean, before running make again.
You should always be able to compile the Quantum ESPRESSO suite of programs without
having to edit any of the generated files. However you may have to tune configure by specifying
appropriate environment variables and/or command-line options. Usually the tricky part is to
get external libraries recognized and used: see Sec.2.5 for details and hints. In many cases, you
may simply edit file [Link].
10
Note that F90 is an “historical” name – we actually use Fortran 2008 – and that it should
be used only together with option --disable-parallel. This is because the value of F90 must
be consistent with the parallel Fortran compiler which is determined by configure and stored
in the MPIF90 variable.
For example, the following command line:
instructs configure to use mpif90 as Fortran compiler with flags -O2 -assume byterecl, gcc
as C compiler with flags -O3, and to link with flag -static. Note that the value of FFLAGS must
be quoted, because it contains spaces. NOTA BENE: passing the complete path to compilers
(e.g., F90=/path/to/f90xyz) may lead to obscure errors during compilation.
11
--enable-parallel compile for parallel (MPI) execution if possible (yes)
--enable-openmp compile for OpenMP execution if possible (no)
--enable-static produce static executables, arger but more portable (no)
--enable-shared produce objects that are suitable for shared libraries (no)
--enable-debug compile with debug flags (no)
--enable-pedantic compile with gfortran pedantic flags on (no)
--enable-signals enable signal trapping (no)
--enable-exit-status enable returning exit status (no)
and the following optional packages:
--with-fox Use official FoX library instead of built-in replacement (default:no)
--with-scalapack (yes|no|intel) Use scalapack if available (default:yes)
Use intel to force Intel MPI and BLACS (obsolescent)
--with-elpa-include Specify full path of ELPA include and modules headers (no)
--with-elpa-lib Specify full path of the ELPA library (no)
--with-elpa-version Specify ELPA API version: 2015 for ELPA releases 2015.x
and 2016.05; 2016 for ELPA releases 2016.11, 2017.x and
2018.05; 2018 for ELPA releases 2018.11 and beyond (2018)
--with-hdf5 (no | yes | <path>)
Compile HDF5 support (no). If “yes”, configure assumes a
valid v. >= 1.8.16 HDF5 installation with h5cc and h5fc in the
default executable search path. If <path> is specified, it must be the
root folder of a standalone hdf5 installation.
--with-hdf5-libs Specify the link options and libraries needed to link HDF5, if configure
fails to detect them. These options are usually composed by many
substrings and must be enclosed into quotes.
--with-hdf5-include Specify full path the HDF5 include folder containing module and
headers files. Use it if configure fails to find the include folder.
--with-libxc Enable support for the libxc library (no)
--with-libxc-prefix directory where libxc is installed
--with-libxc-include directory where libxc Fortran headers reside
12
--with-cuda=value enable compilation of GPU-accelerated subroutines.
value should point the path where the CUDA toolkit
is installed, e.g. $NVHPC CUDA HOME
--with-cuda-cc=value sets the compute capabilities for the compilation
of the accelerated subroutines.
value must be consistent with the hardware and the
NVidia driver installed on the workstation or on the
compute nodes of the HPC facility (default: 35)
--with-cuda-runtime=value (optional) sets the version of the CUDA toolkit used
for the compilation of the accelerated code.
value must be consistent with the
CUDA Toolkit installed on the workstation
or available on the compute nodes of the HPC facility.
--with-cuda-mpi=value yes enables the usage of CUDA-aware MPI library.
Beware: if you have no fast inter-GPU communications, e.g.,
NVlink or Infiniband RDMA, you may get a crash at run time.
Important for optimal parallel performances (default: no).
--enable-nvtx=value enable NVTX profiling (for developers, default: no).
To modify or extend configure (advanced users only!), see the Wiki pages on GitLab:
[Link]
2.5 Libraries
2.5.1 BLAS and LAPACK
Quantum ESPRESSO needs the BLAS and LAPACK mathematical libraries. As a rule,
one should always use vendor-specific optimized BLAS and LAPACK, such as e.g. those found
in Intel’s MKL. They often yield huge performance gains with respect to compiled libraries.
configure always try to locate the best mathematical libraries.
If optimized BLAS and LAPACK are not available, Quantum ESPRESSO automatically
downloads and compiles them. Another option is to try the ATLAS library: [Link]
Note that ATLAS is not a complete replacement for LAPACK: it contains all of the BLAS, plus
13
the LU code, plus the full storage Cholesky code. Follow the instructions in the ATLAS distri-
butions to produce a full LAPACK replacement. Also note that the ATLAS project appears
to be dead.
Sergei Lisenkov reported success and good performances with optimized BLAS by Kazushige
Goto. The library is now available under an open-source license: see the GotoBLAS2 page at
[Link]
2.5.2 FFT
The FFTXlib package of Quantum ESPRESSO contains a copy of an old FFTW library, but
also supports the newer FFTW3 library and some vendor-specific FFT libraries. It is strongly
recommanded to use FFT’s from an optimized library, such as e.g. those contained in Intel
MKL. configure will first search for vendor-specific FFT libraries; if none is found, it will
search for an external FFTW v.3 library; if none is found, it will fall back to the internal copy
of FFTW. configure will add the appropriate preprocessing options:
to DFLAGS in the [Link] file. If you edit [Link] manually, please note that one and only
one among the mentioned preprocessing option must be set.
If you have MKL libraries, you may either link FFTW3 from MKL, or use DFTI (recom-
mended).
2.5.4 HDF5
The HDF5 library ([Link] v.1.8.16 or later, can be
used to perform binary I/O using the HDF5 format.
If compiling the HDF5 library from sources, attention must be paid to pass options:
--enable-fortran, --enable-fortran2003, and --enable-parallel (see below), to the configure
script of HDF5 (not of Quantum ESPRESSO).
To use HDF5 is usually sufficient to specify the path to the fortran compiler wrapper for
HDF5 (h5fc of h5pfc with the --with-hdf5= option of configure. If the wrapper is in the de-
fault path, just use --with-hdf5=yes. The configure script is usually able to extract the linker
options and the include directory path from the output of the wrapper. If it fails, the user can
provide configure options --with-hdf5-libs=<options> and --with-hdf5-include=<path>
14
for the linker options and include path respectively. These options are often needed when using
the HDF5 packages provided by many LINUX distributions. In this case you may first try the
--with-hdf5=yes option. If it fails, just type command h5fc --show (or h5pfc if you are
using parallel HDF5): the command will print out the linker and include options to be passed
manually to the configure script.
The configure script is able to determine whether one is linking to a serial or parallel HDF5
library, and will set the flag -D HDF5 SERIAL in the [Link] file accordingly.
If this still fails, you may set some or all of the * LIBS variables manually and retry. For
example:
Beware that in this case, configure will blindly accept the specified value, and won’t do any
check or extra search.
15
2.6.1 Linking in Quantum ESPRESSO
Once installed libxc, the linking with Quantum ESPRESSO can be enabled directly through
the configuration script by adding the two switches --with-libxc and --with-libxc-prefix,
e.g.:
By adding the first switch only an automatic search for the libxc folder will be attempted, but
its success is not guaranteed. It is always preferable to specify the second switch too. Optionally,
a third switch can be added, namely --with-libxc-include=’/path/to/libxc/include’,
which specifies the path to the Fortran headers (usually it is not necessary).
Alternatively, the link to libxc can be activated after the configuration of Quantum
ESPRESSO by modifying the [Link] file in the main folder in this way:
since the f03 interfaces are no longer available. They have been restored in following releases.
Version 5.0.0 is still usable, but, before compiling Quantum ESPRESSO, a string replacement
is necessary, namely ‘xc f03’ must be replaced with ‘xc f90’ everywhere in the XClib folder.
With CMake: when executing cmake in the build folder, add the following flags:
If cmake is not able to find the package you may need to do this: in Quantum ESPRESSO
main folder open [Link] and, inside the block if(QE\_ENABLE\_LIBXC), near line
583, add this:
set(ENV{PKG_CONFIG_PATH} "$ENV{PKG\_CONFIG\_PATH}:/path/to/libxc/pkgconfig")
Note for versions newer than 5.0.0: libxc enforces the Fermi hole curvature by default,
which might lead to inexact results and convergence problems when using some MGGA func-
tionals in Quantum ESPRESSO. This should be switched off by adding the --disable-fhc
flag when compiling libxc.
16
2.6.2 Usage
In order to use libxc functionals, you can enforce them from input by including the input dft
string in the system namelist. Starting from v7.0 of Quantum ESPRESSO the only allowed
notation for DFTs that include Libxc terms is the index one. For example, to use the libxc
version of the PBE functional (both exchange and correlation):
input_dft = ‘XC-000I-000I-101L-130L-000I-000I’
The letters I or L next to each ID stand for Internal and libxc. This is equivalent to the old
full-name notation:
input_dft = ‘gga_x_pbe gga_c_pbe’ ***OLD***
The order must be the usual one, namely LDA exchange, LDA correlation, GGA exchange,
GGA correlation, MGGA exchange, MGGA correlation. libxc exchange+correlation function-
als can be put in the exchange or in the correlation slot with no difference.
The reason why the full-name notation has been disabled is to eliminate the risk of overlaps
among different names (occurring especially when combinations of internal and libxc DFTs
are used).
The complete list of libxc functionals and IDs is available at: [Link]
Combinations of Quantum ESPRESSO and libxc functionals are allowed in PW, but some
attention has to be paid to their reciprocal compatibility (see section below).
For example, the internal exchange term of PBE together with the correlation term of PBE in
libxc is obtained by:
input_dft = ‘XC-001I-000I-003I-130L-000I-000I’
which corresponds to the old:
input_dft = ‘sla pbx gga_c_pbe’ ***OLD***
Note that when using GGA internal functionals you must always specify the LDA term too,
while it is not the case for the libxc ones.
Abbreviations are allowed when zero tails are present. The above example is still valid by
putting:
input_dft = ‘XC-001I-000I-003L-130L’
since no MGGA terms are present.
Non-local terms can be included by just adding their name after the index notation, for example:
input_dft=‘XC-001i-004i-013i-vdw2’
17
on the cases, therefore a good check of the chosen functionals is recommended before doing
expensive runs.
Some functionals in libxc incorporate the exchange part and the correlation one into one term
only (e.g. the ones that include the ‘ xc’ kind label in their name). In these cases the whole
functional is formally treated as ‘correlation only’ by Quantum ESPRESSO. This does not
imply any loss of information.
External parameters. Several functionals in libxc depend on one or more external pa-
rameters. Some of these can be recovered inside Quantum ESPRESSO, some others are
not directly available. In all these cases a direct intervention on the Quantum ESPRESSO
source code might be necessary in order to be able to properly use such functionals. However
two routines have been defined in the XC library of Quantum ESPRESSO that ease the task
of setting and recovering the external parameters in libxc:
• get libxc ext param: this function receives as input the ID of the libxc functional and
the index of the chosen parameter and returns its value. If the parameter has not been
set before it returns its default value.
• set libxc ext param: this routine receives as input the index of the functional family-
type (from 1 to 6: lda-exch, lda-corr, gga-exch, ...), the index of the chosen libxc param-
eter and the value to set it to.
In order to see the available parameters for a given libxc functional and their corresponding
indexes, the xc infos.x program is available in XClib folder. For more details see Sec. 2.6.5.
The two routines can be called almost anywhere in Quantum ESPRESSO, however, as any
other XClib setting routine, they must be declared through the xc lib module.
Without setting the external parameters inside the code, their default value will be assumed.
This could lead to results different from the expectations.
In any case, when external parameters are needed by the chosen functionals, a warning message
will appear in the output of Quantum ESPRESSO. An example of Libxc parameter setting
can be found in the xclib test.f90 code (see below).
Functionals with partial output. A few libxc functional routines provides the potential
but not the energy. These functionals are available in Quantum ESPRESSO for all the
families and their output energy is set to zero.
MGGA Functionals that depend on the Laplacian of the density. At present such
functionals are formally usable in Quantum ESPRESSO , but their dependency on the
Laplacian is ignored and the corresponding output term of the potential is set to zero. Since
the Laplacian of the density is computable in Quantum ESPRESSO, they might be fully
exploited with a limited intervention on the code.
18
Other functionals. Besides exchange ( x), correlation ( c) and exchange plus correlation
( xc), a fourth kind of functionals is available in libxc, the kinetic functionals ( k). At present,
they are not usable in Quantum ESPRESSO.
2.6.5 XC test
A testing program, xclib test.x, for the XClib library of Quantum ESPRESSO is available.
The program is available for LDA, GGA and MGGA functionals (both QE and Libxc). It also
tests the potential derivatives for LDA (dmxc) and GGA (dgcxc).
Another small program, xc infos.x, is available in the XClib folder starting from v6.8. It
receives as input the name of any DFT usable in Quantum ESPRESSO (both internal and
libxc) and provides infos about their family, type, external parameters, limitations, references,
etc.
See XClib/[Link] file for further details on each of the two programs.
2.7 Compilation
The compiled codes can run with any input: almost all variables are dinamically allocated at
run time. Only a few variables have fixed dimensions, set in file Modules/parameters.f90:
These values should work for the vast majority of cases. In case you need more atomic types
or more k-points, edit this file and recompile.
At your choice, you may compile the complete Quantum ESPRESSO suite of programs
(with make all), or only some specific programs. All executables are linked in main bin
directory. make with no arguments yields ain updated list of valid compilation targets.
For the setup of the GUI, refer to the PWgui-X.Y.Z /INSTALL file, where X.Y.Z stands for
the version number of the GUI (should be the same as the general version number). If you are
using sources from the git repository, see the GUI/README file instead.
If make refuses for some reason to download additional packages, manually download them
into subdirectory archive/, not unpacking or uncompressing them, and try make again. Also
see Sec.(2.1).
19
2.8.1 Test-suite
Automated tests give a ”pass/fail” answer. All tests run quickly (less than a minute each at
most), but they are not meant to be realistic, just to test a specific case. Many features are
tested but only for the following codes: pw.x, cp.x, ph.x, epw.x, hp.x. Instructions for the
impatient:
cd test-suite
make [NPROCS=X] run-tests
where the square brackets mean that what is inside is optional, X is the number of processors
(for a parallel build: do not set X to more than 1 for a serial build!).
Instructions for all others: go to the test-suite/ directory, read the README file, or at
least, type make. You may need to edit the [Link] shells, defining variables PARA PREFIX
and PARA POSTFIX (see below for their meaning).
2.8.2 Examples
There are many examples and reference data for almost every piece of Quantum ESPRESSO,
but you have to manually inspect the results.
In order to use examples, you should edit file environment variables, setting the following
variables as needed.
BIN DIR: directory where executables reside
PSEUDO DIR: directory where pseudopotential files reside
TMP DIR: directory to be used as temporary storage area
The default values of BIN DIR and PSEUDO DIR should be fine, unless you have installed
things in nonstandard places. TMP DIR must be a directory where you have read and write
access to, with enough available space to host the temporary files produced by the example
runs, and possibly offering high I/O performance (i.e., don’t use an NFS-mounted directory).
NOTA BENE: do not use a directory containing other data: the examples will clean it!
If you have compiled the parallel version of Quantum ESPRESSO (this is the default
if parallel libraries are detected), you will usually have to specify a launcher program (such
as mpirun or mpiexec) and the number of processors: see Sec.3 for details. In order to do
that, edit again the environment variables file and set the PARA PREFIX and PARA POSTFIX
variables as needed. Parallel executables will be run by a command like this:
$PARA_PREFIX pw.x $PARA_POSTFIX -i [Link] > [Link]
For example, if the command line is like this (as for an IBM SP):
mpirun -np 4 pw.x -i [Link] > [Link]
you should set PARA PREFIX="mpirun -np 4", PARA POSTFIX="". Furthermore, if your ma-
chine does not support interactive use, you must run the commands specified above through
the batch queuing system installed on that machine. Ask your system administrator for instruc-
tions. For execution using OpenMP on N threads, use PARA PREFIX="env OMP NUM THREADS=N
... ".
To run an example, go to the corresponding directory (e.g. PW/examples/example01) and
execute:
20
./run_example
This will create a subdirectory results/, containing the input and output files generated by
the calculation. Some examples take only a few seconds to run, while others may require up to
several minutes.
In each example’s directory, the reference/ subdirectory contains verified output files,
that you can check your results against. You may get slightly different reesults on different
machines, in particular if different FFT dimensions are automatically selected. For this reason,
a plain diff of your results against the reference data doesn’t work, or at least, it requires human
inspection of the results.
The example scripts stop if an error is detected. You should look inside the last written
output file to understand why.
21
2.9.2 Linux PC’s
Both AMD and Intel CPUs, 32-bit and 64-bit, are supported and work, either in 32-bit emu-
lation and in 64-bit mode. 64-bit executables can address a much larger memory space than
32-bit executable, but there is no gain in speed. Beware: the default integer type for 64-bit
machine is typically 32-bit long. You should be able to use 64-bit integers as well, but it is not
guaranteed to work and will not give any advantage anyway.
It is usually convenient to create semi-statically linked executables (with only libc, libm,
libpthread dynamically linked). If you want to produce a binary that runs on different machines,
compile it on the oldest machine you have (i.e. the one with the oldest version of the operating
system).
Currently, configure supports, and Quantum ESPRESSO works with, not-too-old and
not-too-buggy versions of gfortran, Intel (ifx, ifort), NVidia (nvfortran), AMD (AOCC v.5),
ARM (armflang), Cray (ftn) compilers.
Linux PCs with Intel compiler (ifx, ifort) If configure doesn’t find the compiler, or
if you get Error loading shared libraries at run time, you may have forgotten to execute the
script that sets up the correct PATH and library path. Unless your system manager has done
this for you, you should execute the appropriate script – located in the directory containing the
compiler executable – in your initialization files. Consult the documentation provided by Intel.
Linux PCs with MKL libraries On Intel CPUs it is very convenient to use Intel MKL
libraries (freely available together with the Intel compiler at [Link]
They can be used also with non-Intel compilers. With gfortran, one has to link -lmkl gf lp64
instead of -lmkl intel lp64 (configure should take care of it).
configure properly detects MKL libraries, as long as the $MKLROOT environment vari-
able is set in the current shell. Normally this environment variable is set by sourcing the
environment script provided by Intel.
By default the non-threaded version of MKL is linked, unless option configure --with-openmp
is specified. In case of trouble, refer to the following web page to find the correct way to link
MKL:
[Link]
For parallel (MPI) execution on multiprocessor (SMP) machines, set the environment vari-
able OMP NUM THREADS to 1 unless you know how to run MPI+OpenMP. See Sec.3 for
more info on this and on the difference between MPI and OpenMP parallelization.
Linux PCs with AMD processors For AMD CPUs there are optimized libraries called
AOCL, AMD Optimizing CPU Libraries, bundled with the AOCC v.5 compiler, freely available
from AMD.
“ Recently I played around with some AMD EPYC cpus and the bad thing is that I also
saw some strange numbers when using libflame/aocl 2.1. (...) Since version 2020 the MKL
performs rather well when using AMD cpus, however, if you want to get the best performance
you have to additionally set:
export MKL_DEBUG_CPU_TYPE=5
22
which gives an additional 10-20% speedup with MKL 2020, while for earlier versions the speedup
is greater than 200%. [...] Another note, there seem to be problems using FFTW interface of
MKL with AMD cpus. To get around this problem, one has to additionally set
export MKL_CBWR=AUTO
“ (Info by Tobias Klöffel, Feb. 2020)
• configure tries to locate libraries (both mathematical and parallel libraries) in the usual
places with usual names, but if they have strange names or strange locations, you will
have to rename/move them, or to instruct configure to find them. If MPI libraries are
not found, parallel compilation is disabled.
• configure tests that the compiler and the libraries are compatible (i.e. the compiler may
link the libraries without conflicts and without missing symbols). If they aren’t and the
compilation fails, configure will revert to serial compilation.
Apart from such problems, Quantum ESPRESSO compiles and works on all non-buggy,
properly configured hardware and software combinations. In some cases you may have to
recompile MPI libraries: not all MPI installations contain support for the Fortran compiler of
your choice (or for any Fortran compiler at all).
If Quantum ESPRESSO does not work for some reason on a PC cluster, try first if
it works in serial execution. A frequent problem with parallel execution is that Quantum
ESPRESSO does not read from standard input, due to the configuration of MPI libraries: see
Sec.3.5. If you are dissatisfied with the performances in parallel execution, see Sec.3 and in
particular Sec.3.5.
23
provided by PGI (the configure of FoX fails: use script install/build fox with [Link] to
manually compile FoX).
Another option: use MinGW/MSYS. Download the installer from [Link]
install MinGW, MSYS, gcc and gfortran. Start a shell window; run ”./configure”; edit [Link];
uncommenting the second definition of TOPDIR (the first one introduces a final ”/” that Win-
dows doesn’t like); run ”make”. Note that on some Windows the code fails when checking that
tmp dir is writable, for unclear reasons.
Another option is Cygwin, a UNIX environment which runs under Windows: see
[Link]
2.9.5 Mac OS
Mac OS-X machines with gfortran, and possibly other compilers as well, should in principle
work, but ”your mileage may vary”, depending upon the specific software stack you are using.
Parallel compilation with OpenMPI should also work.
Gfortran information and binaries for Mac OS-X here: [Link]
If you get an error like
• Install homebrew
• Using homebrew install gcc (11.2.0), open-mpi (4.1.1 2), fftw3 (3.3.10), and veclibfort
(0.4.2 7)
To configure QE:
24
2.9.6 Cray machines
Cray machines may be tricky: ”... despite what people can imagine, every CRAY machine
deployed can have different environment. For example on the machine I usually use for tests
[...] I do have to unload some modules to make QE running properly. On another CRAY [...]
there is also Intel compiler as option and the system is slightly different compared to the other.”
(info by Filippo Spiga)
./configure ARCH=craype should work for recent Cray machines. This selects the ftn
compiler, that typically uses the crayftn compiler but may also use other ones, depending
upon the site and personal environment. ftn v.15.0.1 and later should compile QE properly.
Some compiler versions may however run into problems like these for ftn v.14.0.3:
25
3 Parallelism
3.1 Understanding Parallelism
Two different parallelization paradigms are currently implemented in Quantum ESPRESSO:
1. Message-Passing (MPI). A copy of the executable runs on each CPU; each copy lives in a
different world, with its own private set of data, and communicates with other executables
only via calls to MPI libraries. MPI parallelization requires compilation for parallel
execution, linking with MPI libraries, execution using a launcher program (depending
upon the specific machine). The number of CPUs used is specified at run-time either as
an option to the launcher or by the batch queue system.
2. OpenMP. A single executable spawn subprocesses (threads) that perform in parallel spe-
cific tasks. OpenMP can be implemented via compiler directives (explicit OpenMP) or
via multithreading libraries (library OpenMP). Explicit OpenMP require compilation for
OpenMP execution; library OpenMP requires only linking to a multithreading version
of the mathematical libraries. The number of threads is specified at run-time in the
environment variable OMP NUM THREADS.
1. a launcher program such as mpirun or mpiexec, with the appropriate options (if any);
Items 1) and 2) are machine- and installation-dependent, and may be different for interactive
and batch execution. Note that large parallel machines are often configured so as to disallow
interactive execution: if in doubt, ask your system administrator. Item 3) also depend on your
specific configuration (shell, execution path, etc). Item 4) is optional but it is very important
for good performances. We refer to the next section for a description of the various possibilities.
26
3.3 Parallelization levels
In Quantum ESPRESSO several MPI parallelization levels are implemented, in which both
calculations and data structures are distributed across processors. Processors are organized in
a hierarchy of groups, which are identified by different MPI communicators level. The groups
hierarchy is as follow:
• images: Processors can then be divided into different ”images”, each corresponding to a
different self-consistent or linear-response calculation, loosely coupled to others.
• pools: each image can be subpartitioned into ”pools”, each taking care of a group of
k-points.
• bands: each pool is subpartitioned into ”band groups”, each taking care of a group of
Kohn-Sham orbitals (also called bands, or wavefunctions). Especially useful for calcula-
tions with hybrid functionals.
• PW: orbitals in the PW basis set, as well as charges and density in either reciprocal or real
space, are distributed across processors. This is usually referred to as ”PW paralleliza-
tion”. All linear-algebra operations on array of PW / real-space grids are automatically
and effectively parallelized. 3D FFT is used to transform electronic wave functions from
reciprocal to real space and vice versa. The 3D FFT is parallelized by distributing planes
of the 3D grid in real space to processors (in reciprocal space, it is columns of G-vectors
that are distributed to processors).
• tasks: In order to allow good parallelization of the 3D FFT when the number of processors
exceeds the number of FFT planes, FFTs on Kohn-Sham states are redistributed to
“task” groups so that each group can process several wavefunctions at the same time.
Alternatively, when this is not possible, a further subdivision of FFT planes is performed.
Note however that not all parallelization levels are implemented in all codes.
When a communicator is split, the MPI process IDs in each sub-communicator remain
ordered. So for instance, for two images and 2n MPI processes, image 0 contains IDs 0, 1, ..., n−
1, image 1 contains IDs n, n + 1, .., 2n − 1.
27
About communications Images and pools are loosely coupled: inter-processors communi-
cation between different images and pools is modest. Processors within each pool are instead
tightly coupled and communications are significant. This means that fast communication hard-
ware is needed if your pool extends over more than a few processors on different nodes.
Choosing parameters : To control the number of processors in each group, command line
switches: -nimage, -npools, -nband, -ntg, -ndiag or -northo (shorthands, respectively: -ni,
-nk, -nb, -nt, -nd) are used. As an example consider the following command line:
mpirun -np 4096 ./neb.x -ni 8 -nk 2 -nt 4 -nd 144 -i [Link]
This executes a NEB calculation on 4096 processors, 8 images (points in the configuration space
in this case) at the same time, each of which is distributed across 512 processors. k-points are
distributed across 2 pools of 256 processors each, 3D FFT is performed using 4 task groups (64
processors each, so the 3D real-space grid is cut into 64 slices), and the diagonalization of the
subspace Hamiltonian is distributed to a square grid of 144 processors (12x12).
Default values are: -ni 1 -nk 1 -nt 1 ; nd is set to 1 if ScaLAPACK is not compiled, it
is set to the square integer smaller than or equal to the number of processors of each pool.
Massively parallel calculations For very large jobs (i.e. O(1000) atoms or more) or for
very long jobs, to be run on massively parallel machines (e.g. IBM BlueGene) it is crucial to use
in an effective way all available parallelization levels: on linear algebra (requires compilation
with ELPA and/or ScaLAPACK), on ”task groups” (requires run-time option ”-nt N”), and
mixed MPI-OpenMP (requires OpenMP compilation: configure–enable-openmp). Without a
judicious choice of parameters, large jobs will find a stumbling block in either memory or CPU
requirements. Note that I/O may also become a limiting factor.
28
The directory for data is specified in input variables outdir and prefix (the former can
be specified as well in environment variable ESPRESSO TMPDIR): outdir/[Link]. A
copy of pseudopotential files is also written there. If some processor cannot access the data
directory, the pseudopotential files are read instead from the pseudopotential directory specified
in input data. Unpredictable results may follow if those files are not the same as those in the
data directory!
IMPORTANT: Avoid I/O to network-mounted disks (via NFS) as much as you can! Ideally
the scratch directory outdir should be a modern Parallel File System. If you do not have any,
you can use local scratch disks (i.e. each node is physically connected to a disk and writes to
it) but you may run into trouble anyway if you need to access your files that are scattered in
an unpredictable way across disks residing on different nodes.
You can use input variable disk io to vary the amount of I/O done by pw.x. The default
value is disk io=’low’, so the code will store wavefunctions into RAM and not on disk during
the calculation. Specify disk io=’medium’ only if you have too many k-points and you run
into trouble with memory; choose disk io=’none’ if you do not need to keep final data files.
Trouble with input files Input files should be plain ASCII text. The presence of CRLF
line terminators (may appear as ˆM, Control-M, characters at the end of lines), tabulators, or
non-ASCII characters (e.g. non-ASCII quotation marks, that at a first glance may look the
same as the ASCII character) is a frequent source of trouble. Typically, this happens with
files coming from Windows or produced with ”smart” editors. Verify with command file and
convert with command iconv if needed.
Some implementations of the MPI library have problems with input redirection in parallel.
This typically shows up under the form of mysterious errors when reading data. If this happens,
use the option -i (or -in, -inp, -input), followed by the input file name. Example:
29
Of course the input file must be accessible by the processor that must read it (only one processor
reads the input file and subsequently broadcasts its contents to all other processors).
Apparently the LSF implementation of MPI libraries manages to ignore or to confuse even
the -i/in/inp/input mechanism that is present in all Quantum ESPRESSO codes. In this
case, use the -i option of [Link] to provide an input file.
30