Skip to content

Farnam

Louise

Farnam is a shared-use resource for the Yale School of Medicine (YSM). It consists of a variety of compute nodes networked over ethernet and mounts several shared filesystems.

The Farnam Cluster is named for Louise Whitman Farnam, the first woman to graduate from the Yale School of Medicine, class of 1916.


Upcoming Farnam Retirement

After more than six years in service, we will be retiring the Farnam HPC cluster in mid 2023. Farnam is being replaced with the new HPC cluster, McCleary. For more information and updates see the McCleary announcement page.

Access the Cluster

Once you have an account, the cluster can be accessed via ssh or through the Open OnDemand web portal.

System Status and Monitoring

For system status messages and the schedule for upcoming maintenance, please see the system status page. For a current node-level view of job activity, see the cluster monitor page (VPN only).

Partitions and Hardware

Farnam is made up of several kinds of compute nodes. We group them into (sometimes overlapping) Slurm partitions meant to serve different purposes. By combining the --partition and --constraint Slurm options you can more finely control what nodes your jobs can run on.

Job Submission Rate Limits

Job submissions are limited to 200 jobs per hour. See the Rate Limits section in the Common Job Failures page for more info.

Public Partitions

See each tab below for more information about the available common use partitions.

Use the general partition for most batch jobs. This is the default if you don't specify one with --partition.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the general partition are subject to the following limits:

Limit Value
Maximum job time limit 30-00:00:00
Maximum CPUs per group 400
Maximum memory per group 2.50T
Maximum CPUs per user 200
Maximum memory per user 1280G

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
68 E5-2660_v3 20 117 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, common

Use the interactive partition to jobs with which you need ongoing interaction. For example, exploratory analyses or debugging compilation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=06:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the interactive partition are subject to the following limits:

Limit Value
Maximum job time limit 1-00:00:00
Maximum CPUs per user 20
Maximum memory per user 256G
Maximum running jobs per user 2

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
83 E5-2660_v3 20 117 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, common

Use the transfer partition to stage data for your jobs to and from cluster storage.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the transfer partition are subject to the following limits:

Limit Value
Maximum job time limit 1-00:00:00
Maximum running jobs per user 2
Maximum CPUs per job 1

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
2 E5-2660_v3 20 117 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, common

Use the gpu partition for jobs that make use of GPUs. You must request GPUs explicitly with the --gpus option in order to use them. For example, --gpus=gtx1080ti:2 would request 2 GeForce GTX 1080Ti GPUs per node.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the gpu partition are subject to the following limits:

Limit Value
Maximum job time limit 2-00:00:00
Maximum CPUs per user 32
Maximum GPUs per user 12
Maximum memory per user 256G

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) GPU Type GPUs/Node vRAM/GPU (GB) Node Features
1 6240 36 370 a100 4 40 cascadelake, avx2, avx512, 6240, doubleprecision, common, bigtmp, a100
2 5222 8 181 rtx5000 4 16 cascadelake, avx2, avx512, 5222, doubleprecision, common, bigtmp, rtx5000
5 E5-2637_v4 8 119 gtx1080ti 4 11 broadwell, avx2, E5-2637_v4, singleprecision, common, gtx1080ti
2 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, common, k80

Use the gpu_devel partition to debug jobs that make use of GPUs, or to develop GPU-enabled code.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=00:10:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the gpu_devel partition are subject to the following limits:

Limit Value
Maximum job time limit 02:00:00
Maximum submitted jobs per user 1

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) GPU Type GPUs/Node vRAM/GPU (GB) Node Features
1 E5-2623_v4 8 57 gtx1080ti 4 11 broadwell, avx2, E5-2623_v4, singleprecision, common, gtx1080ti

Use the bigmem partition for jobs that have memory requirements other partitions can't handle.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the bigmem partition are subject to the following limits:

Limit Value
Maximum job time limit 3-00:00:00
Maximum CPUs per user 32
Maximum memory per user 1505G
Maximum running jobs per user 2
Maximum nodes per job 1

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
3 6240 36 1505 cascadelake, avx2, avx512, 6240, nogpu, common, bigtmp
2 E7-4809_v3 32 1505 haswell, avx2, E7-4809_v3, nogpu, common

Use the scavenge partition to run preemptable jobs on more resources than normally allowed. For more information about scavenge, see the Scavenge documentation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the scavenge partition are subject to the following limits:

Limit Value
Maximum job time limit 7-00:00:00
Maximum CPUs per user 800
Maximum memory per user 5T

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) GPU Type GPUs/Node vRAM/GPU (GB) Node Features
1 6240 36 181 cascadelake, avx2, avx512, 6240, nogpu, standard, pi, bigtmp
1 6240 36 370 a100 4 40 cascadelake, avx2, avx512, 6240, doubleprecision, common, bigtmp, a100
3 6240 36 1505 cascadelake, avx2, avx512, 6240, nogpu, common, bigtmp
5 E5-2637_v4 8 119 gtx1080ti 4 11 broadwell, avx2, E5-2637_v4, singleprecision, common, gtx1080ti
1 E5-2623_v4 8 57 gtx1080ti 4 11 broadwell, avx2, E5-2623_v4, singleprecision, common, gtx1080ti
16 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi
85 E5-2660_v3 20 117 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, common
2 E7-4809_v3 32 1505 haswell, avx2, E7-4809_v3, nogpu, common
3 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, pi, k80
2 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, common, k80

Use the scavenge_gpu partition to run preemptable jobs on more GPU resources than normally allowed. For more information about scavenge, see the Scavenge documentation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the scavenge_gpu partition are subject to the following limits:

Limit Value
Maximum job time limit 2-00:00:00
Maximum GPUs per user 64

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) GPU Type GPUs/Node vRAM/GPU (GB) Node Features
1 6240 36 370 a100 4 40 cascadelake, avx2, avx512, 6240, doubleprecision, common, bigtmp, a100
5 E5-2637_v4 8 119 gtx1080ti 4 11 broadwell, avx2, E5-2637_v4, singleprecision, common, gtx1080ti
1 E5-2623_v4 8 57 gtx1080ti 4 11 broadwell, avx2, E5-2623_v4, singleprecision, common, gtx1080ti
3 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, pi, k80
2 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, common, k80

Private Partitions

With few exceptions, jobs submitted to private partitions are not considered when calculating your group's Fairshare. Your group can purchase additional hardware for private use, which we will make available as a pi_groupname partition. These nodes are purchased by you, but supported and administered by us. After vendor support expires, we retire compute nodes. Compute nodes can range from $10K to upwards of $50K depending on your requirements. If you are interested in purchasing nodes for your group, please contact us.

PI Partitions (click to expand)

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_gerstein_gpu partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) GPU Type GPUs/Node vRAM/GPU (GB) Node Features
3 E5-2660_v3 20 119 k80 4 12 haswell, avx2, E5-2660_v3, doubleprecision, pi, k80

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_jadi partition are subject to the following limits:

Limit Value
Maximum job time limit 365-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
3 E5-2660_v3 20 117 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, common

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_kleinstein partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
3 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_krauthammer partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
1 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_ma partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
2 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_ohern partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
3 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_sigworth partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
1 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_sindelar partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
1 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_townsend partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
5 E5-2660_v3 20 119 haswell, avx2, E5-2660_v3, nogpu, standard, oldest, pi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=1-00:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_tucci partition are subject to the following limits:

Limit Value
Maximum job time limit 14-00:00:00

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count CPU Type CPUs/Node Memory/Node (GiB) Node Features
1 6240 36 181 cascadelake, avx2, avx512, 6240, nogpu, standard, pi, bigtmp

Public Datasets

We host datasets of general interest in a loosely organized directory tree in /gpfs/gibbs/data:

├── alphafold-2.3
├── alphafold-2.2 (deprecated)
├── alphafold-2.0 (deprecated)
├── annovar
│   └── humandb
├── cryoem
├── db
│   ├── annovar
│   ├── blast
│   ├── busco
│   └── Pfam
└── genomes
    ├── 1000Genomes
    ├── 10xgenomics
    ├── Aedes_aegypti
    ├── Chelonoidis_nigra
    ├── Danio_rerio
    ├── Drosophila_melanogaster
    ├── hisat2
    ├── Homo_sapiens
    ├── Mus_musculus
    ├── PhiX
    └── Saccharomyces_cerevisiae

If you would like us to host a dataset or questions about what is currently available, please contact us.

Storage

Farnam has access to a number of GPFS filesystems. /gpfs/ysm is Farnam's primary filesystem where Home, Project, and Scratch60 directories are located. For more details on the different storage spaces, see our Cluster Storage documentation.

You can check your current storage usage & limits by running the getquota command. Your ~/project and ~/scratch60 directories are shortcuts. Get a list of the absolute paths to your directories with the mydirectories command. If you want to share data in your Project or Scratch directory, see the permissions page.

For information on data recovery, see the Backups and Snapshots documentation.

Warning

Files stored in scratch60 are purged if they are older than 60 days. You will receive an email alert one week before they are deleted. Artificial extension of scratch file expiration is forbidden without explicit approval from the YCRC. Please purchase storage if you need additional longer term storage.

Partition Root Directory Storage File Count Backups Snapshots
home /gpfs/ysm/home 125GiB/user 500,000 Yes >=2 days
project /gpfs/ysm/project 1TiB/group, increase to 4TiB on request 5,000,000 No >=2 days
scratch60 /gpfs/ysm/scratch60 20TiB/group 15,000,000 No >=2 days

Last update: February 24, 2023