Web Portal (Open OnDemand)
Open OnDemand (OOD) is platform for accessing the clusters that only requires a web browser. This web-portal provides a shell, file browser, and graphical interface for certain apps (like Jupyter or MATLAB).
If you access Open OnDemand installed on YCRC clusters from off campus, you will need to first connect to the Yale VPN.
Open OnDemand is available on each cluster using your NetID credentials (CAS login). The Yale CAS login is configured with the DUO authentication. We recommend that you click "Remember me for 90 days" when you are prompted to choose an authentication menthod for DUO. This will simplified the login process.
The above four URLs are also called cluster OOD URLs. They are available to any user with a research account (also called a lab account) on the clusters. Your research account is the same as your NetID.
OOD for Courses
Each course on the YCRC clusters has its own URL to access OOD on the cluster. The URL is unique to each course and is also called course OOD. Course OODs all follow the same naming convention: coursename.ycrc.yale.edu. 'courename' is an abbreviated name given to the course by YCRC. Students must use the course URL to log in to OOD. They will with their NetID to log in but work under their student account on the cluster while they are in OOD.
Course OOD and cluster OOD have different URLs, even if they use the same physical machine. Student accounts can only log in to OOD through a course OOD URL, and a regular account (same as your NetID) can only log in through the cluster OOD URL.
If you only have a student account, but try to log in through the cluster OOD URL, you will get an error in the browser:
Error -- can't find user for cpsc424_test Run 'nginx_stage --help' to see a full list of available command line options.
Additional information about course OOD can be found at academic support.
On login you will see the OOD dashboard.
Along the top are pull-down menus for various Apps, including File Managers, Job Composer, a Shell, a list of Interactive Apps, etc.
The file browser is a graphical interface to manage, upload, and download files from the clusters. You can use the built-in file editor to view and edit files from your browser without having to download and upload scripts.
You can also drag-and-drop to download and upload files and directories, and move files between directories using this interface.
Customize Favorite Paths
Users are allowed to customize favorite paths in the file manager. Using the scripts below to add, remove, and list customized paths:
ood_add_path ood_remove_path ood_list_path
ood_add_pathfrom a shell command line, it will prompt you to add one path at a time, until you type 'n' to discontinue. All the paths added by you will be shown in the OOD pull-down menu for the file manager, as well as the left pane when the file manager is opened.
ood_remove_path allows you to remove any of the paths added by you and
ood_list_path will list all the paths added by you.
After you have customized the path configuration from a shell, go to the OOD dashbaord and click
Restart Web Server on the top menu bar to make the change effective immediately.
You can launch a traditional command-line interface to the cluster using the Shell pull-down menu. This opens a terminal in a web-browser that you can use in the exact same way as when logging into the cluster via SSH.
This is a convenient way to access the clusters when you don't have access to an ssh client or do not have your ssh keys.
We have deployed a selection of common graphical programs as Interactive Apps on Open OneDemand. Currently, we have apps for Remote Desktop, MATLAB, Mathematica, RStudio Desktop, RStudio Server, and Jupyter Notebook, etc.
You are limited to 4 interactive app instances (of any type) at one time. Additional instances will be rejected until you delete older open instances. Closing the window does not terminate the interactive app job. To terminate the job, click the "Delete" button in your "My Interactive Apps" page in the web portal.
Occasionally, it is helpful to use a graphical interface to explore data or run certain programs. In the past your options were to use VNC or X11 forwarding. These tools can be complex to setup or suffer from reduced performance. The Remote Desktop app from OOD simplifies the configuration of a VNC desktop session on a compute node. The MATLAB, Mathematica, and RStudio Desktop Apps are special versions of this app. To get started choose Remote Desktop (or another desktop app) from the Interactive Apps menu on the dashboard.
Use the form to request resources and decide what partition your job should run on. Use
interactive on Milgram) or your lab's partition.
Once you launch the job, you will be presented with a notification that your job has been queued. Depending on the resources requested, you may need to wait for a bit. When the job starts you will see the option to launch the Remote Desktop:
Note you can share a view only link for your session if you would like to share your screen. After you click on Launch Remote Desktop, a standard desktop interface will open in a new tab.
In some browsers, you may have to use a special text box to copy and paste from the Remote Desktop App. Click the arrow on the left side of your window for a menu, then click the clipboard icon to get access to your Remote Desktop's clipboard.
One of the most common uses of Open OnDemand is the Jupyter interface for Python and R. You can choose either Jupyter Notebook or Jupyter Lab. By default, this app will try to launch Jupyter Notebook, unless the
Start JupyterLab checkbox is selected.
Make sure that you chose the right Conda environment for you from the drop-down menu. If you have not yet set one up, follow our instructions on how to create a new one. After specifying the required resources (number of CPUs/GPUs, amount of RAM, etc.), you can submit the job. When it launches you can open the standard Jupyter interface where you can start working with notebooks.
The Jupyter root directory is set to your Home when started. Project and Scratch can be accessed via their respective symlinks in Home. If you want to access a directory that cannot be acessed through your home directory, for example Gibbs, you need to create a symlink to that directory in your home directory.
ycrc_default conda environment will be automatically built when you select it for the first time from Jupyter. You can also build your own Jupyter and make it available to OOD:
module load miniconda conda create -n env_name jupyter jupyter-lab ycrc_conda_env.sh update
ycrc_default will not be updated by OOD automatically. It must be updated by the user manually. To update
ycrc_default, run the following command from a shell command line:
module load miniconda conda update -n ycrc_default jupyter jupyter-lab
Change User R Package Path
To change the default path where packages installed by the user are stored, you need to add the following line of code in your
Configure the Graphic Device
When you plot in a RStudio session, you may encounter the following error:
Error in RStudioGD() : Shadow graphics device error: r error 4 (R code execution error) In addition: Warning message: In grDevices:::png("/tmp/RtmpcRxRaB/4v3450e3627g4432fa27f516348657267.png", : unable to open connection to X11 display ''
To fix the problem, you need to configure your RStudio session to use
Cairo for plotting.
You can do it in your code as follows:
Alternatively, you can put the above code in
.Rprofile in your home directory and the option will be picked up automatically.
If RStudio becomes slow to respond or completely stops responding, please stop the RStudio session and then run the following script at a shell command line:
This will remove any temporary files created by RStudio and allow it to start anew.
An OOD session is started and then completed immediately
- Check if your quota is full
- Reset your
.bash_profileto their original contents (you can backup the startup files before resetting them. Add the changes back one at a time to see if one or more of the changes would affect OOD from starting properly)
- Remove the default module collection file
$HOME/.lmod.d/default.cluster-rhel8(cluster is one of the following: grace, mccleary) or
Remote Desktop (or MATLAB, Mathematica, etc) cannot be started properly
- Make sure there is no initialization left by
conda initin your
.bashrc. Clean it with
sed -i.bak -ne '/# >>> conda init/,/# <<< conda init/!p' ~/.bashrc
dbus-launchand make sure you see the following output:
[pl543@grace1 ~]$ which dbus-launch /usr/bin/dbus-launch
Jupyter cannot be started properly
- If you are trying to launch
jupyter-notebook, make sure it is available in your jupyter conda environment:
(ycrc_default)[pl543@grace1 ~]$ which jupyter-notebook /gpfs/gibbs/project/support/pl543/conda_envs/ycrc_default/bin/jupyter-notebook
- If you are trying to launch
jupyter-lab, make sure it is available in your jupyter conda environment:
(ycrc_default)[pl543@grace1 ~]$ which jupyter-lab /gpfs/gibbs/project/support/pl543/conda_envs/ycrc_default/bin/jupyter-notebook
RStudio with Conda R
If you see
NOT_FOUND in "Conda R Environment", it means your Conda R environment has not been properly installed. You may need to reinstall your Conda R environment and make sure
r-base r-essentials are both included.
RStudio Server does not respond
If you encounter a grey screen after clicking the "Connect to RStudio Server" button, please stop the RStudio session and run
clean-rstudio.sh at a shell command line.