Grace

Grace is a shared-use resource for the Faculty of Arts and Sciences (FAS). It consists of a variety of compute nodes networked over low-latency InfiniBand and mounts several shared filesystems.

The Grace cluster is is named for the computer scientist and United States Navy Rear Admiral Grace Murray Hopper, who received her Ph.D. in Mathematics from Yale in 1934.

Operating System Upgrade

During the August 2023 maintenance, the operating system on Grace was upgraded from Red Hat 7 to Red Hat 8. For more information, see our Grace Operating System Upgrade page.

Access the Cluster

Once you have an account, the cluster can be accessed via ssh or through the Open OnDemand web portal.

System Status and Monitoring

For system status messages and the schedule for upcoming maintenance, please see the system status page. For a current node-level view of job activity, see the cluster monitor page (VPN only).

Partitions and Hardware

Grace is made up of several kinds of compute nodes. We group them into (sometimes overlapping) Slurm partitions meant to serve different purposes. By combining the --partition and --constraint Slurm options you can more finely control what nodes your jobs can run on.

Job Submission Limits

You are limited to 4 interactive app instances (of any type) at one time. Additional instances will be rejected until you delete older open instances. For OnDemand jobs, closing the window does not terminate the interactive app job. To terminate the job, click the "Delete" button in your "My Interactive Apps" page in the web portal.
Job submissions are limited to 200 jobs per hour. See the Rate Limits section in the Common Job Failures page for more info.

Public Partitions

See each tab below for more information about the available common use partitions.

day

Use the day partition for most batch jobs. This is the default if you don't specify one with --partition.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the day partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum CPUs per group	`2500`
Maximum CPUs per user	`1000`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
66	6342	48	488	icelake, avx512, 6342, nogpu, standard, common, bigtmp
72	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, common, bigtmp
130	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, common, bigtmp

devel

Use the devel partition to jobs with which you need ongoing interaction. For example, exploratory analyses or debugging compilation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the devel partition are subject to the following limits:

Limit	Value
Maximum job time limit	`06:00:00`
Maximum CPUs per user	`4`
Maximum memory per user	`32G`
Maximum submitted jobs per user	`1`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	6126	24	174	skylake, avx512, 6126, nogpu, standard, common
4	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

week

Use the week partition for jobs that need a longer runtime than day allows.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the week partition are subject to the following limits:

Limit	Value
Maximum job time limit	`7-00:00:00`
Maximum CPUs per group	`252`
Maximum CPUs per user	`108`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
18	6342	48	488	icelake, avx512, 6342, nogpu, standard, common, bigtmp

transfer

Use the transfer partition to stage data for your jobs to and from cluster storage.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the transfer partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum running jobs per user	`2`
Maximum CPUs per job	`1`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	7642	8	237	epyc, 7642, nogpu, standard, common

gpu

Use the gpu partition for jobs that make use of GPUs. You must request GPUs explicitly with the --gpus option in order to use them. For example, --gpus=gtx1080ti:2 would request 2 GeForce GTX 1080Ti GPUs per node.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the gpu partition are subject to the following limits:

Limit	Value
Maximum job time limit	`2-00:00:00`
Maximum GPUs per user	`24`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
2	6326	32	984	a100	4	80	icelake, avx512, 6326, doubleprecision, bigtmp, common, gpu, a100-80g
11	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, common
4	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, common, v100
3	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, common, bigtmp, rtx2080ti
1	6240	36	361	a100	4	40	cascadelake, avx512, 6240, doubleprecision, bigtmp, common, a100
2	5222	8	181	rtx5000	4	16	cascadelake, avx512, 5222, doubleprecision, common, bigtmp, rtx5000
2	6136	24	90	v100	2	16	skylake, avx512, 6136, doubleprecision, common, bigtmp, v100
6	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, common, p100, oldest

gpu_devel

Use the gpu_devel partition to debug jobs that make use of GPUs, or to develop GPU-enabled code.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the gpu_devel partition are subject to the following limits:

Limit	Value
Maximum job time limit	`06:00:00`
Maximum CPUs per user	`10`
Maximum GPUs per user	`2`
Maximum submitted jobs per user	`2`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
1	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, common
2	6240	36	166	rtx3090	4	24	cascadelake, avx512, 6240, doubleprecision, bigtmp, common, rtx3090
1	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, common, v100
1	6240	36	361	a100	4	40	cascadelake, avx512, 6240, doubleprecision, bigtmp, common, a100
4	5222	8	181	rtx5000	4	16	cascadelake, avx512, 5222, doubleprecision, common, bigtmp, rtx5000
2	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, common, bigtmp, rtx2080ti

bigmem

Use the bigmem partition for jobs that have memory requirements other partitions can't handle.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the bigmem partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum CPUs per user	`40`
Maximum memory per user	`4000G`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	6234	16	1505	cascadelake, avx512, nogpu, 6234, common, bigtmp
3	6240	36	1505	cascadelake, avx512, 6240, nogpu, common, bigtmp
4	6346	32	3936	cascadelake, avx512, 6346, common, nogpu, bigtmp

mpi

Use the mpi partition for tightly-coupled parallel programs that make efficient use of multiple nodes. See our MPI documentation if your workload fits this description.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --exclusive --mem=90112

Job Limits

Jobs submitted to the mpi partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum nodes per group	`64`
Maximum nodes per user	`64`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
128	6136	24	88	hdr, skylake, avx512, 6136, nogpu, standard, common, bigtmp

scavenge

Use the scavenge partition to run preemptable jobs on more resources than normally allowed. For more information about scavenge, see the Scavenge documentation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the scavenge partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum CPUs per user	`10000`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
1	6326	32	984	l40s	4	48	icelake, avx512, 6326, pi, standard, bigtmp, l40s
50	6342	48	488				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
84	6342	48	488				icelake, avx512, 6342, nogpu, standard, common, bigtmp
6	6342	48	2000				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	6326	32	984	a100	4	80	icelake, avx512, 6326, doubleprecision, bigtmp, common, gpu, a100-80g
2	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
2	6326	32	469	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
11	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, common
72	8268	48	356				cascadelake, avx512, 8268, nogpu, standard, common, bigtmp
72	8268	48	356				cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
87	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
135	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, common, bigtmp
4	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, common, v100
3	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, common, bigtmp, rtx2080ti
1	6240	36	361	a100	4	40	cascadelake, avx512, 6240, doubleprecision, bigtmp, common, a100
2	6234	16	1505				cascadelake, avx512, nogpu, 6234, common, bigtmp
2	5222	8	181	rtx5000	4	16	cascadelake, avx512, 5222, doubleprecision, common, bigtmp, rtx5000
20	8260	96	181				cascadelake, avx512, 8260, nogpu, pi
3	6240	36	1505				cascadelake, avx512, 6240, nogpu, common, bigtmp
1	6326	32	1001	a100	4	80	cascadelake, avx512, 6326, doubleprecision, bigtmp, pi, a100-80g
1	6254	36	370	rtx2080ti	8	11	cascadelake, avx512, 6254, singleprecision, pi, bigtmp, rtx2080ti
2	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, pi, v100
4	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, pi, bigtmp, rtx2080ti
2	6240	36	180	rtx3090	4	24	cascadelake, avx512, 6240, doubleprecision, bigtmp, pi, rtx3090
8	6240	36	370				cascadelake, avx512, 6240, nogpu, pi, bigtmp
4	6346	32	3936				cascadelake, avx512, 6346, common, nogpu, bigtmp
3	6234	16	1505				cascadelake, avx512, nogpu, 6234, pi, bigtmp
16	6136	24	90				edr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp
3	6142	32	181				skylake, avx512, 6142, nogpu, standard, pi, bigtmp
12	6136	24	90				hdr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp
4	6136	24	88				hdr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp
9	6136	24	181	p100	4	16	skylake, avx512, 6136, doubleprecision, pi, p100
2	5122	8	181	rtx2080	4	8	skylake, avx512, 5122, singleprecision, pi, rtx2080
2	6136	24	90	v100	2	16	skylake, avx512, 6136, doubleprecision, common, bigtmp, v100
4	6136	24	90				hdr, skylake, avx512, 6136, nogpu, pi, common, bigtmp
1	6136	24	749				skylake, avx512, 6136, nogpu, pi, bigtmp
74	E5-2660_v4	28	245				broadwell, E5-2660_v4, nogpu, standard, pi, oldest
6	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, common, p100, oldest
2	E7-4820_v4	40	1505				broadwell, E7-4820_v4, nogpu, pi, oldest
1	E5-2637_v4	8	119	gtx1080ti	4	11	broadwell, E5-2637_v4, singleprecision, pi, bigtmp, gtx1080ti, oldest
1	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, pi, p100, oldest

scavenge_gpu

Use the scavenge_gpu partition to run preemptable jobs on more GPU resources than normally allowed. For more information about scavenge, see the Scavenge documentation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the scavenge_gpu partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum GPUs per user	`30`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
1	6326	32	984	l40s	4	48	icelake, avx512, 6326, pi, standard, bigtmp, l40s
2	6326	32	984	a100	4	80	icelake, avx512, 6326, doubleprecision, bigtmp, common, gpu, a100-80g
2	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
2	6326	32	469	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
11	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, common
4	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, common, v100
3	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, common, bigtmp, rtx2080ti
1	6240	36	361	a100	4	40	cascadelake, avx512, 6240, doubleprecision, bigtmp, common, a100
2	5222	8	181	rtx5000	4	16	cascadelake, avx512, 5222, doubleprecision, common, bigtmp, rtx5000
1	6326	32	1001	a100	4	80	cascadelake, avx512, 6326, doubleprecision, bigtmp, pi, a100-80g
1	6254	36	370	rtx2080ti	8	11	cascadelake, avx512, 6254, singleprecision, pi, bigtmp, rtx2080ti
2	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, pi, v100
4	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, pi, bigtmp, rtx2080ti
2	6240	36	180	rtx3090	4	24	cascadelake, avx512, 6240, doubleprecision, bigtmp, pi, rtx3090
9	6136	24	181	p100	4	16	skylake, avx512, 6136, doubleprecision, pi, p100
2	5122	8	181	rtx2080	4	8	skylake, avx512, 5122, singleprecision, pi, rtx2080
2	6136	24	90	v100	2	16	skylake, avx512, 6136, doubleprecision, common, bigtmp, v100
6	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, common, p100, oldest
1	E5-2637_v4	8	119	gtx1080ti	4	11	broadwell, E5-2637_v4, singleprecision, pi, bigtmp, gtx1080ti, oldest
1	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, pi, p100, oldest

scavenge_mpi

Use the scavenge_mpi partition to run preemptable jobs on more MPI resources than normally allowed. For more information about scavenge, see the Scavenge documentation.

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --exclusive --mem=90112

Job Limits

Jobs submitted to the scavenge_mpi partition are subject to the following limits:

Limit	Value
Maximum job time limit	`1-00:00:00`
Maximum nodes per group	`64`
Maximum nodes per user	`64`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
128	6136	24	88	hdr, skylake, avx512, 6136, nogpu, standard, common, bigtmp

Private Partitions

With few exceptions, jobs submitted to private partitions are not considered when calculating your group's Fairshare. Your group can purchase additional hardware for private use, which we will make available as a pi_groupname partition. These nodes are purchased by you, but supported and administered by us. After vendor support expires, we retire compute nodes. Compute nodes can range from $10K to upwards of $50K depending on your requirements. If you are interested in purchasing nodes for your group, please contact us.

PI Partitions (click to expand)

pi_anticevic

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_anticevic partition are subject to the following limits:

Limit	Value
Maximum job time limit	`100-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
15	6342	48	488				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	6342	48	2000				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	6326	32	206	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
20	E5-2660_v4	28	245				broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_balou

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_balou partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
14	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
9	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
26	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_berry

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_berry partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
1	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_chem_chase

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=3840

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_chem_chase partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
8	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
1	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, pi, bigtmp, rtx2080ti

pi_cowles

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_cowles partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`
Maximum CPUs per user	`120`
Maximum nodes per user	`5`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
9	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp

pi_econ_io

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_econ_io partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
6	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_econ_lp

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_econ_lp partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
6	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
7	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
5	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
1	6234	16	1505	cascadelake, avx512, nogpu, 6234, pi, bigtmp

pi_esi

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_esi partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`
Maximum CPUs per user	`648`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
36	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_fedorov

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=3840

Job Limits

Jobs submitted to the pi_fedorov partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
6	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
12	6136	24	90	hdr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp
4	6136	24	90	hdr, skylake, avx512, 6136, nogpu, pi, common, bigtmp

pi_gelernter

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_gelernter partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
1	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_hammes_schiffer

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=3840

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_hammes_schiffer partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
6	8268	48	356				cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
8	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
1	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, pi, bigtmp, rtx2080ti
16	6136	24	90				edr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp
2	5122	8	181	rtx2080	4	8	skylake, avx512, 5122, singleprecision, pi, rtx2080
1	6136	24	749				skylake, avx512, 6136, nogpu, pi, bigtmp
1	E5-2637_v4	8	119	gtx1080ti	4	11	broadwell, E5-2637_v4, singleprecision, pi, bigtmp, gtx1080ti, oldest

pi_hodgson

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_hodgson partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_holland

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_holland partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
8	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_howard

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_howard partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_jorgensen

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_jorgensen partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
3	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_kim_theodore

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_kim_theodore partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
1	6234	16	1505	cascadelake, avx512, nogpu, 6234, pi, bigtmp

pi_korenaga

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_korenaga partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
3	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp

pi_lederman

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_lederman partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
1	6254	36	1505	rtx4000,rtx8000,v100	4,2,2	8,48,16	cascadelake, avx512, 6254, pi, bigtmp, rtx8000

pi_levine

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=1952

Job Limits

Jobs submitted to the pi_levine partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
20	8260	96	181	cascadelake, avx512, 8260, nogpu, pi

pi_lora

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=3840

Job Limits

Jobs submitted to the pi_lora partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
5	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
4	6136	24	88	hdr, skylake, avx512, 6136, nogpu, standard, pi, bigtmp

pi_mak

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_mak partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_manohar

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_manohar partition are subject to the following limits:

Limit	Value
Maximum job time limit	`180-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
6	6342	48	488				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	8268	48	356				cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
4	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
8	E5-2660_v4	28	245				broadwell, E5-2660_v4, nogpu, standard, pi, oldest
2	E7-4820_v4	40	1505				broadwell, E7-4820_v4, nogpu, pi, oldest
1	E5-2660_v4	28	245	p100	1	16	broadwell, E5-2660_v4, doubleprecision, pi, p100, oldest

pi_mingarelli

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_mingarelli partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
4	6342	48	2000	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
6	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
4	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, common, bigtmp

pi_ohern

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_ohern partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
2	6342	48	488				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
8	8268	48	356				cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
2	6240	36	181				cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp
9	6136	24	181	p100	4	16	skylake, avx512, 6136, doubleprecision, pi, p100
3	E5-2660_v4	28	245				broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_owen_miller

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_owen_miller partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp
1	6234	16	1505	cascadelake, avx512, nogpu, 6234, pi, bigtmp
5	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_padmanabhan

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_padmanabhan partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
3	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_panda

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_panda partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
2	6342	48	488				icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	6326	32	469	a5000	4	24	icelake, avx512, 6326, doubleprecision, a5000, pi
1	6326	32	1001	a100	4	80	cascadelake, avx512, 6326, doubleprecision, bigtmp, pi, a100-80g
1	6254	36	370	rtx2080ti	8	11	cascadelake, avx512, 6254, singleprecision, pi, bigtmp, rtx2080ti
2	6240	36	370	v100	4	16	cascadelake, avx512, 6240, doubleprecision, pi, v100
3	6240	36	181	rtx2080ti	4	11	cascadelake, avx512, 6240, singleprecision, pi, bigtmp, rtx2080ti

pi_poland

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_poland partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
8	6240	36	370	cascadelake, avx512, 6240, nogpu, pi, bigtmp
9	E5-2660_v4	28	245	broadwell, E5-2660_v4, nogpu, standard, pi, oldest

pi_polimanti

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_polimanti partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
2	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, pi, bigtmp

pi_seto

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_seto partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
3	6142	32	181	skylake, avx512, 6142, nogpu, standard, pi, bigtmp

pi_spielman

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_spielman partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp

pi_sweeney

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_sweeney partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
2	6240	36	180	rtx3090	4	24	cascadelake, avx512, 6240, doubleprecision, bigtmp, pi, rtx3090

pi_tsmith

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_tsmith partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
2	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
1	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp

pi_vaccaro

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_vaccaro partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
1	6342	48	488	icelake, avx512, 6342, nogpu, standard, pi, bigtmp
1	6240	36	181	cascadelake, avx512, 6240, nogpu, standard, common, bigtmp

pi_ying_rex

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

GPU jobs need GPUs!

Jobs submitted to this partition do not request a GPU by default. You must request one with the --gpus option.

Job Limits

Jobs submitted to the pi_ying_rex partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	GPU Type	GPUs/Node	vRAM/GPU (GB)	Node Features
1	6326	32	984	l40s	4	48	icelake, avx512, 6326, pi, standard, bigtmp, l40s

pi_zhu

Request Defaults

Unless specified, your jobs will run with the following options to salloc and sbatch options for this partition.

--time=01:00:00 --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=5120

Job Limits

Jobs submitted to the pi_zhu partition are subject to the following limits:

Limit	Value
Maximum job time limit	`28-00:00:00`

Available Compute Nodes

Requests for --cpus-per-task and --mem can't exceed what is available on a single compute node.

Count	CPU Type	CPUs/Node	Memory/Node (GiB)	Node Features
12	8268	48	356	cascadelake, avx512, 8268, nogpu, standard, pi, bigtmp

Storage

Grace has access to a number of filesystems. /vast/palmer hosts Grace's home and scratch directories and /gpfs/gibbs hosts project directories and most additional purchased storage allocations. For more details on the different storage spaces, see our Cluster Storage documentation.

You can check your current storage usage & limits by running the getquota command. Your ~/project and ~/palmer_scratch directories are shortcuts. Get a list of the absolute paths to your directories with the mydirectories command. If you want to share data in your Project or Scratch directory, see the permissions page.

For information on data recovery, see the Backups and Snapshots documentation.

Warning

Files stored in palmer_scratch are purged if they are older than 60 days. You will receive an email alert one week before they are deleted. Artificial extension of scratch file expiration is forbidden without explicit approval from the YCRC. Please purchase storage if you need additional longer term storage.

Partition	Root Directory	Storage	File Count	Backups	Snapshots
home	`/vast/palmer/home.grace`	125GiB/user	500,000	Yes	>=2 days
project	`/gpfs/gibbs/project`	1TiB/group, increase to 4TiB on request	5,000,000	No	>=2 days
scratch	`/vast/palmer/scratch`	10TiB/group	15,000,000	No	No

Last update: February 7, 2024