Skip to content

Web Portal (Open OnDemand)

Open OnDemand (OOD) is platform for accessing the clusters that only requires a web browser. This web-portal provides a shell, file browser, and graphical interface for certain apps (like Jupyter or MATLAB).

Access

If you access Open OnDemand installed on YCRC clusters from off campus, you will need to first connect to the Yale VPN.

Open OnDemand is available on each cluster using your NetID credentials (CAS login). The Yale CAS login is configured with the DUO authentication. We recommend that you click "Remember me for 90 days" when you are prompted to choose an authentication menthod for DUO. This will simplified the login process.

Cluster OOD site
Grace ood-grace.ycrc.yale.edu
McCleary ood-mccleary.ycrc.yale.edu
Milgram ood-milgram.ycrc.yale.edu

The above four URLs are also called cluster OOD URLs. They are available to any user with a research account (also called a lab account) on the clusters. Your research account is the same as your NetID.

OOD for Courses

Each course on the YCRC clusters has its own URL to access OOD on the cluster. The URL is unique to each course and is also called course OOD. Course OODs all follow the same naming convention: coursename.ycrc.yale.edu. 'courename' is an abbreviated name given to the course by YCRC. Students must use the course URL to log in to OOD. They will with their NetID to log in but work under their student account on the cluster while they are in OOD.

Course OOD and cluster OOD have different URLs, even if they use the same physical machine. Student accounts can only log in to OOD through a course OOD URL, and a regular account (same as your NetID) can only log in through the cluster OOD URL.

Warning

If you only have a student account, but try to log in through the cluster OOD URL, you will get an error in the browser: 

Error -- can't find user for cpsc424_test
Run 'nginx_stage --help' to see a full list of available command line options.
Use the URL for your course OOD will resolve the problem.

Additional information about course OOD can be found at academic support.

The Dashboard

On login you will see the OOD dashboard.

welcome

Along the top are pull-down menus for various Apps, including File Managers, Job Composer, a Shell, a list of Interactive Apps, etc.

File Browser

file_browser

The file browser is a graphical interface to manage, upload, and download files from the clusters. You can use the built-in file editor to view and edit files from your browser without having to download and upload scripts.

You can also drag-and-drop to download and upload files and directories, and move files between directories using this interface.

Customize Favorite Paths

Users are allowed to customize favorite paths in the file manager. Using the scripts below to add, remove, and list customized paths:

  ood_add_path
  ood_remove_path
  ood_list_path
When you run ood_add_path from a shell command line, it will prompt you to add one path at a time, until you type 'n' to discontinue. All the paths added by you will be shown in the OOD pull-down menu for the file manager, as well as the left pane when the file manager is opened.

ood_remove_path allows you to remove any of the paths added by you and ood_list_path will list all the paths added by you.

After you have customized the path configuration from a shell, go to the OOD dashbaord and click Develop->Restart Web Server on the top menu bar to make the change effective immediately.

Shell

You can launch a traditional command-line interface to the cluster using the Shell pull-down menu. This opens a terminal in a web-browser that you can use in the exact same way as when logging into the cluster via SSH.

This is a convenient way to access the clusters when you don't have access to an ssh client or do not have your ssh keys.

Interactive Apps

We have deployed a selection of common graphical programs as Interactive Apps on Open OneDemand. Currently, we have apps for Remote Desktop, MATLAB, Mathematica, RStudio Desktop, RStudio Server, and Jupyter Notebook, etc.

Warning

You are limited to 4 interactive app instances (of any type) at one time. Additional instances will be rejected until you delete older open instances. Closing the window does not terminate the interactive app job. To terminate the job, click the "Delete" button in your "My Interactive Apps" page in the web portal.

Remote Desktop

Occasionally, it is helpful to use a graphical interface to explore data or run certain programs. In the past your options were to use VNC or X11 forwarding. These tools can be complex to setup or suffer from reduced performance. The Remote Desktop app from OOD simplifies the configuration of a VNC desktop session on a compute node. The MATLAB, Mathematica, and RStudio Desktop Apps are special versions of this app. To get started choose Remote Desktop (or another desktop app) from the Interactive Apps menu on the dashboard.

Use the form to request resources and decide what partition your job should run on. Use devel (interactive on Milgram) or your lab's partition.

remote_desktop

Once you launch the job, you will be presented with a notification that your job has been queued. Depending on the resources requested, you may need to wait for a bit. When the job starts you will see the option to launch the Remote Desktop:

starting

Note you can share a view only link for your session if you would like to share your screen. After you click on Launch Remote Desktop, a standard desktop interface will open in a new tab.

Copy/Paste

In some browsers, you may have to use a special text box to copy and paste from the Remote Desktop App. Click the arrow on the left side of your window for a menu, then click the clipboard icon to get access to your Remote Desktop's clipboard.

clipboard

Jupyter

One of the most common uses of Open OnDemand is the Jupyter interface for Python and R. You can choose either Jupyter Notebook or Jupyter Lab. By default, this app will try to launch Jupyter Notebook, unless the Start JupyterLab checkbox is selected.

jupyter_form

Make sure that you chose the right Conda environment for you from the drop-down menu. If you have not yet set one up, follow our instructions on how to create a new one. After specifying the required resources (number of CPUs/GPUs, amount of RAM, etc.), you can submit the job. When it launches you can open the standard Jupyter interface where you can start working with notebooks.

Root directory

The Jupyter root directory is set to your Home when started. Project and Scratch can be accessed via their respective symlinks in Home. If you want to access a directory that cannot be acessed through your home directory, for example Gibbs, you need to create a symlink to that directory in your home directory.

ycrc_default

The ycrc_default conda environment will be automatically built when you select it for the first time from Jupyter. You can also build your own Jupyter and make it available to OOD:

module load miniconda
conda create -n env_name jupyter jupyter-lab
ycrc_conda_env.sh update

Once created, ycrc_default will not be updated by OOD automatically. It must be updated by the user manually. To update ycrc_default, run the following command from a shell command line: 

module load miniconda
conda update -n  ycrc_default jupyter jupyter-lab

VSCode Server

Visual Studio Code is a popular development tool that is widely used by our researchers. While there are several extensions that allow users to connect to remote servers over SSH, these are imperfect and often drop connection. Additionally, these remote sessions connect to the clusters' login nodes, where resources are limited. We have developed an application for OOD that launches VS Code in a job on a compute node and opens in a web-browser. This application is called code_server and is available on all clusters.

RStudio Server

RStudio Server works with R from an R module or from an R Conda environment. Selected R modules on the cluster (ususally the two most recent versions installed on the cluster) are available in the user form. If you want to use Conda R, you need to run ycrc_conda_env.sh update to have it listed in the user form.

Change User R Package Path

To change the default path where packages installed by the user are stored, you need to add the following line of code in your $HOME/.bashrc:

export R_LIBS_USER=path_to_your_local_r_packages

Configure the Graphic Device

When you plot in a RStudio session, you may encounter the following error:

Error in RStudioGD() : 
  Shadow graphics device error: r error 4 (R code execution error)
In addition: Warning message:
In grDevices:::png("/tmp/RtmpcRxRaB/4v3450e3627g4432fa27f516348657267.png",  :
  unable to open connection to X11 display ''

To fix the problem, you need to configure your RStudio session to use Cairo for plotting. You can do it in your code as follows: 

options(bitmapType='cairo')

Alternatively, you can put the above code in .Rprofile in your home directory and the option will be picked up automatically.

Clean RStudio

If RStudio becomes slow to respond or completely stops responding, please stop the RStudio session and then run the following script at a shell command line:

ycrc_clean_rstudio.sh

This will remove any temporary files created by RStudio and allow it to start anew.

Run RStudio in Remote Desktop

While we don't generally encourage our users to run a production R code in RStudio, there are cases that it could be benificial. For example, when a user needs to monitor the R code's progress constantly.

RStudio Server is not user friendly for long-running R code. When your CAS session timeout, you won't be able to reconnect while the code is running. You'll need to wait until the code finishes before you can connect to the same session again.

If you need to monitor your R code's progress continuously within the same R session without concerns about disconnection, you can run RStudio Desktop within a Remote Desktop environment.

Using R module with RStudio Desktop

First, start a Remote Desktop instance in OOD. From the terminal in the Remote Desktop, run the following commands:

module load R
module load RStudio
rstudio
Using R Conda with RStudio Desktop

If you want to use R in a Conda environment, start a Remote Desktop instance in OOD first. From the terminal in the Remote Desktop, please don't load the modules for R and RStudio. Instead, please install 'rstudio-desktop' into your R Conda environment if you haven't done so, and then call rstudio. 

module load miniconda
conda activate my_r_env
conda install rstudio-desktop
rstudio

Troubleshoot OOD

An OOD session is started and then completed immediately

  1. Check if your quota is full
  2. Reset your .bashrc and .bash_profile to their original contents (you can backup the startup files before resetting them. Add the changes back one at a time to see if one or more of the changes would affect OOD from starting properly)
  3. Remove the default module collection file $HOME/.lmod.d/default.cluster-rhel8 (cluster is one of the following: grace, mccleary) or $HOME/.lmod.d/default.milgram-rhel7 for Milgram.

Remote Desktop (or MATLAB, Mathematica, etc) cannot be started properly

  1. Make sure there is no initialization left by conda init in your .bashrc. Clean it with 
    sed -i.bak -ne '/# >>> conda init/,/# <<< conda init/!p' ~/.bashrc
  2. Run dbus-launch and make sure you see the following output: 
    [pl543@grace1 ~]$ which dbus-launch
/usr/bin/dbus-launch

Jupyter cannot be started properly

  1. If you are trying to launch jupyter-notebook, make sure it is available in your jupyter conda environment: 
    (ycrc_default)[pl543@grace1 ~]$ which jupyter-notebook
/gpfs/gibbs/project/support/pl543/conda_envs/ycrc_default/bin/jupyter-notebook
  2. If you are trying to launch jupyter-lab, make sure it is available in your jupyter conda environment: 
    (ycrc_default)[pl543@grace1 ~]$ which jupyter-lab
/gpfs/gibbs/project/support/pl543/conda_envs/ycrc_default/bin/jupyter-notebook

RStudio with Conda R

If you see NOT_FOUND in "Conda R Environment", it means your Conda R environment has not been properly installed. You may need to reinstall your Conda R environment and make sure r-base r-essentials are both included.

RStudio Server does not respond

If you encounter a grey screen after clicking the "Connect to RStudio Server" button, please stop the RStudio session and run clean-rstudio.sh at a shell command line.

Last update: February 22, 2024