# Gaussian
## Gaussian short introduction
---
1. Make [gaussian.slurm](gaussian.slurm) batch script:
#!/bin/bash
#SBATCH --job-name=Job_Name
#SBATCH --mem-per-cpu=1GB
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=24
#SBATCH -t 1-00:00:00
#SBATCH --partition=common
#SBATCH --no-requeue
module load green/all
module load Gaussian/16.c01
SCRATCH=/state/partition1/$SLURM_JOB_ID
export GAUSS_SCRDIR=$SCRATCH
mkdir -p $SCRATCH
g16 -m=24gb -p=24 < job.com > job.log
#Clean after yourself
rm -rf $SCRATCH
2. Copy job-input file [job.com](job.com).
3. Submit the job on **base**:
sbatch gaussian.slurm
***NB!*** _More cores does not mean faster!!! See [Benchmarks](https://hpc.pages.taltech.ee/user-guides/chemistry/gaussian.html#benchmarks-for-parallel-jobs)._
4. Check results using [visualization software](visualization.md).
## Gaussian long version
---
[Gaussian](https://gaussian.com/g16main/) is a general purpose package for calculation of electronic structures. It can calculate properties of molecules (structures, energies, spectroscopic and thermochemical properties, atomic charges, electron affinities and ionization potentials, electrostatic potentials and electron densities etc.) and reactions properties (such as reaction pathways, IRC) using different methods (such as Molecular mechanics, Semi-empirical methods, Hartree-Fock, Density functional, Møller-Plesset perturbation theory, coupled cluster). More about Gaussian features can be sound [here](https://gaussian.com/g16glance/).
### Environment
There are currently two versions of Gaussian available on HPC: g09 (revision C.01) and g16 (revision C.01). Environment is set up by the commands:
module load green/all
module load Gaussian/16.c01
***NB!*** _For gray-ib partition gaussian16.c01.AVX should be loaded, for other partitions - gaussian16.c01.AVX2. More information on partition are in [taltech user-guides](https://hpc.pages.taltech.ee/user-guides/index.html#hardware-specification)._
### Running Gaussian jobs
Gaussian input files are executed by the commands `g09` or `g16` depending on the version of Gaussian used. This command is usually placed in Slurm script.
g16 < job.com > job.log
### Single core calculations
Gaussian by default execute jobs on only a single processor.
***NB!*** _If in a `slurm` script is dedined more processors, they will be reserved, but not utilizes._
### Parallel jobs
#### Single-node (SMP) parallelism
To run multiple processors/cores job a number of cores should be specified. The number of cores can be defined via the `-p` flag (e.g. -p=8) in command line or by adding the `%NprocShared` command into Gaussian input file (e.g. %NprocShared=24). For more information see [Gaussian manual](https://gaussian.com/running/). The number of processors requested should correspond to the number of processors requested in `slurm` script.
Example of `slurm` script for Gray partition:
#!/bin/bash
#SBATCH --job-name=Job_Name
#SBATCH --mem-per-cpu=1GB
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH -t 1-00:00:00
#SBATCH --partition=gray-ib
module load green/all
module load Gaussian/16.c01
SCRATCH=/state/partition1/$SLURM_JOB_ID
export GAUSS_SCRDIR=$SCRATCH
mkdir -p $SCRATCH
g16 < job.com > job.log
rm -rf $SCRATCH
Example of Gaussian input:
%Mem=4GB
%NprocShared=4
%chk=job.chk
#P B3LYP/6-311++G** Opt EmpiricalDispersion=GD3BJ
Job_Name
0,1
C 0.67650 0.42710 0.00022
H 0.75477 1.52537 0.00197
O 1.62208 -0.30498 -0.00037
S -1.01309 -0.16870 0.00021
H -1.58104 1.05112 -0.00371
Example of `slurm` script for Green partition:
#!/bin/bash
#SBATCH --job-name=Job_Name
##SBATCH --mem-per-cpu=1GB
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH -t 1-00:00:00
#SBATCH --partition=common
module load green/all
module load Gaussian/16.c01
SCRATCH=/state/partition1/$SLURM_JOB_ID
export GAUSS_SCRDIR=$SCRATCH
mkdir -p $SCRATCH
g16 -m=4gb -p=4 < job.com > job.log
rm -rf $SCRATCH
Example of Gaussian input:
%chk=job.chk
#P B3LYP/AUG-cc-pVTZ Opt Freq=(NoRaman)
HCOSH trans
0 1
C
H 1 1.1010569
O 1 1.1958524 2 123.6711058
O 1 1.4300000 3 122.8283301 2 -179.9280276
H 4 0.9600000 1 109.5000001 3 179.9228016
#### Memory
The default dynamic memory requested by Gaussian is frequently too small for successful job termination. Herein, if amount of memory requested is insufficient, the job can crash. There is no golden rule for memory requests. Usually, for common calculations (e.g. optimization, frequency etc.) 1-2 GB per 1 CPU is sufficient. This can be done by the `-m` flag in the command line (e.g. -m=48gb) or by adding the `%Mem` command in Gaussian input file (e.g. %Mem=1GB). For more information see [Gaussian manual](https://gaussian.com/running/) and [taltech user-guides](https://hpc.pages.taltech.ee/user-guides/index.html#hardware-specification).
However, there are calculations that require more memory (e.g TD-DFT, large SCF calculations, etc.). Data from a `slurm-JOBID.stat` file can be useful to determine the amount of memory required for a computation. In `slurm-JOBID.stat` file the efficiency of memory utilization is shown.
Bad example:
Memory Utilized: 3.08 GB
Memory Efficiency: 11.83% of 26.00 GB
Good example:
Memory Utilized: 63.12 GB
Memory Efficiency: 98.62% of 64.00 GB
#### Time
Time limits depend on time partition used [taltech user-guides](https://hpc.pages.taltech.ee/user-guides/index.html#hardware-specification). If the calculation time exceeds the time limit requested in the Slurm script, then the job will be killed. Therefore, it is recommended to request more time than is usually needed for calculation.
### Using GPUs
GPUs **are effective** for larger molecules, DFT energies, gradients and frequencies calculations. GPUs **are not effective** for small jobs, MP2 or CCSD calculations.
GPU jobs can be run only on **amp1** or **amp2**. To access **amp** user has to have ssh-keys copied to the **base** ([how to do that](../ssh.md)).
**Amp** can be accessed by command:
ssh -X -Y -J uni-id@base.hpc.taltech.ee uni-id@amp
Each GPU must be controlled by a specific CPU, wherein, CPUs used as GPU controllers do not participate as compute nodes during the calculations.
The GPUs and CPUS used for calculations are specified with the `%GPUCPU` command, where gpu- and cpu-lists are a comma-separated lists, possibly including numerical ranges (e.g., 0-4,6). The corresponding items in the two lists are the GPU and its controlling CPU.
%cpu=0-9
%gpucpu=0-1=0-1
***NB!*** _Note that the controlling CPUs are included in `%CPU` command._
Example of [gaussian-gpu.slurm](gaussian-gpu.slurm) script for **amp**:
#!/bin/bash
#SBATCH --job-name=Job_Name
#SBATCH -t 1:00:00
#SBATCH --no-requeue
#SBATCH --partition=gpu-test
#SBATCH --gres=gpu:A100:2
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=10
#SBATCH --mem=160GB
module use /gpfs/mariana/modules/system
module load green/all
module load Gaussian/16.c02
module use /illukas/software/modules
module load cuda/11.4
SCRATCH=/state/partition1/$SLURM_JOB_ID
export GAUSS_SCRDIR=$SCRATCH
mkdir -p $SCRATCH
g16 job.com > job.log
#Clean after yourself
rm -rf $SCRATCH
Example of Gaussian input [job-gpu.com](job-gpu.com) (bad example, since molecule is small):
%mem=160GB
%cpu=0-9
%gpucpu=0-1=0-1
# wb97xd/cc-pVDZ opt
GPU test
0 1
N 0.15134980 0.09540020 1.45819090
C 0.75130720 -1.21343470 1.83361500
H -0.39763650 0.48328420 2.23924210
H 0.89227330 0.78663430 1.26643050
C 2.08354180 -1.05427080 2.58684570
H 2.46028270 -2.07052410 2.83958100
H 1.89047670 -0.54642730 3.56027240
H 0.94017970 -1.74157910 0.87502690
C -0.27659390 -2.02386680 2.62322330
H -1.22744720 -2.09807790 2.06259160
H 0.09671630 -3.04868580 2.81503150
H -0.48448270 -1.55043170 3.60777900
C 3.18448030 -0.29819020 1.82025510
C 4.47118690 -0.16399430 2.65811660
C 3.50345720 -0.94054930 0.45675520
H 2.82725210 0.74302520 1.61940860
C 5.57163930 0.60375030 1.90697440
H 4.83763400 -1.18683360 2.90651280
H 4.24528280 0.32880900 3.62913610
C 4.59840130 -0.17643230 -0.29951390
H 3.82853470 -1.99290760 0.62773340
H 2.59267050 -0.99453480 -0.16920790
C 5.87589210 -0.03780040 0.54257470
H 6.49233690 0.65984540 2.52562860
H 5.23630520 1.65412070 1.74873850
H 4.81114970 -0.67947290 -1.26682010
H 4.21177720 0.83532620 -0.55856640
H 6.64325110 0.55322730 -0.00130230
H 6.31606730 -1.04822960 0.70571860
#### Allocation of memory
Allocating sufficient amounts of memory for GPU jobs is even more important when for CPU jobs. GPUs can have up to 16 GB of memory, wherein, must be at least an equal amount of memory given to the GPU and each control CPU thread.
Gaussian gives equal shares of memory to each thread, this means that the total memory allocated should be the number of threads times the memory required. Allocating Gaussian 11-12 GB is appropriate for a 16 GB GPU (see example above).
### _Restarting a failed/interrupted calculation_
Killed or failed jobs can be restarted, but for this checkpoint file should be generated via a `%Chk` command within the Gaussian input file. For more information see [Gaussian FAQ](https://gaussian.com/faq2/), [Gaussian restart](https://gaussian.com/restart/) and [Using Gaussian Checkpoint Files](http://www.ccl.net/cca/documents/dyoung/topics-orig/checkpoint.html).
***NB!*** _Checkpoint files are very heavy and are readable only on the machine on which they were generated. After successful completion of the calculation, it is recommended to delete these files._
### How to cite:
- _Gaussian 16 - [https://gaussian.com/citation/](https://gaussian.com/citation/)_
- _Gaussian 09 - Gaussian 09, Revision C.01, Frisch, M.J.; Trucks, G.W.; Schlegel, H.B.; Scuseria, G.E.; Robb, M.A.; Cheeseman, J.R.; Scalmani, G.; Barone, V.; Mennucci, B.; Petersson, G.A.; Nakatsuji, H.; Caricato, M.; Li, X.; Hratchian, H.P.; Izmaylov, A.F.; Bloino, J.; Zheng, G.; Sonnenberg, J.L.; Hada, M.; Ehara, M.; Toyota, K.; Fukuda, R.; Hasegawa, J.; Ishida, M.; Nakajima, T.; Honda, Y.; Kitao, O.; Nakai, H.; Vreven, T.; Montgomery, J.A., Jr.; Peralta, J.E.; Ogliaro, F.; Bearpark, M.; Heyd, J.J.; Brothers, E.; Kudin, K.N.; Staroverov, V.N.; Kobayashi, R.; Normand, J.; Raghavachari, K.; Rendell, A.; Burant, J.C.; Iyengar, S.S.; Tomasi, J.; Cossi, M.; Rega, N.; Millam, N.J.; Klene, M.; Knox, J.E.; Cross, J.B.; Bakken, V.; Adamo, C.; Jaramillo, J.; Gomperts, R.; Stratmann, R.E.; Yazyev, O.; Austin, A. J.; Cammi, R.; Pomelli, C.; Ochterski, J. W.; Martin, R.L.; Morokuma, K.; Zakrzewski, V.G.; Voth, G.A.; Salvador, P.; Dannenberg, J.J.; Dapprich, S.; Daniels, A.D.; Farkas, Ö.; Foresman, J.B.; Ortiz, J.V.; Cioslowski, J.; Fox, D.J. Gaussian, Inc., Wallingford CT, 2009._
### Benchmarks for parallel jobs
Gaussian example benchmarks performed with Gaussian 16 C01 AVX2.
The job had 82 atoms.
