Load Software with Modules
To facilitate the diverse work that happens on the YCRC clusters we compile, install, and manage software packages separately from those installed in standard system directories. We use EasyBuild to build, install, and manage packages. You can access these packages as Lmod modules. The modules involving compiled software are arranged into hierarchical toolchains that make dependencies more consistent when you load multiple modules.
Avoid loading Python or R modules and conda environments. This will almost always break something.
All Available Modules
To list all available modules, run:
Search For Modules
You can search for modules or extensions with
avail. For example, to find and list all Python version 3 modules, run:
module avail python/3
To find any module or extension that mentions python in its name or description, use the command:
module spider python
Get Module Help
You can get a brief description of a module and the url to the software's homepage by running:
module help modulename/version
Load and Unload Modules
module load command modifies your environment so you can use the specified software package(s).
- This command is case-sensitive to module names.
module loadcommand will load dependencies as needed, you don't need to load them separately.
- For batch jobs, add
module loadcommand(s) to your submission script.
For example, to load
2.11.0, find modules with matching toolchain suffixes and run the command:
module load Python/3.8.6-GCCcore-10.2.0 BLAST+/2.11.0-GCCcore-10.2.0
Lmod will add
python and the BLAST commands to your environment. Since both of these modules were built with the
GCCcore/10.2.0 toolchain module, they will not load conflicting libraries. Recall you can see the other modules that were loaded by running
As new versions of software get installed and others are deprecated, the default module version can change over time. It is best practice to note the specific module versions you are using for a project and load those explicitly, e.g.
module load Python/3.8.6-GCCcore-10.2.0 not
module load Python. This makes your work more reproducible and less likely to change unexpectedly in the future.
You can also unload a specific module that you've previously loaded:
module unload R
Or unload all modules at once with:
module purge will alert you to a sticky module that is always loaded called
StdEnv. Avoid unloading
StdEnv unless explicitly told to do so, othewise you will lose some important setup for the cluster you are on.
It can be a pain to enter a long list of modules every time you return to a project. Module collections allow you to create sets of modules to load together. This method is particularly useful if you have two or more module sets that may conflict with one another.
Save a collection of modules by first loading all the modules you want to save together then run:
module save environment_name
environment_name with something more meaningful to you)
Load a collection with
module restore environment_name
To modify a collection:
restore it, make the desired changes by
unloading modules, then
save it to the same name.
To get a list of your collections, run:
ml: A Convinient Tool
Lmod provides a convinient tool called
ml to simplify all of the module commands.
List Module Loaded
ml can be used to replace the
module command. It can take all the sub-commands from
module and works the same way as
ml load Python R ml unload Python ml spider Python ml avail ml whatis Python ml key Python ml purge ml save test ml restore test
To refer to the directory where the software from a module is stored, you can use the environment variable
$EBROOTMODULENAME where MODULENAME is the name of the module in all caps with no spaces. This can be useful for finding the executables, libraries, or readme files that are included with the software:
[netid@node ~ ]$ module load SAMtools [netid@node ~ ]$ echo $EBVERSIONSAMTOOLS 1.11 [netid@node ~ ]$ ls $EBROOTSAMTOOLS bin easybuild include lib lib64 share [netid@node ~ ]$ ls $EBROOTSAMTOOLS/bin ace2sam maq2sam-short psl2sam.pl soap2sam.pl blast2sam.pl md5fa r2plot.lua vcfutils.lua bowtie2sam.pl md5sum-lite sam2vcf.pl wgsim export2sam.pl novo2sam.pl samtools wgsim_eval.pl interpolate_sam.pl plot-ampliconstats samtools.pl zoom2sam.pl maq2sam-long plot-bamstats seq_cache_populate.pl
You can view documentation while on the cluster using the command:
There is even more information at the offical Lmod website.