Department of Statistics

Getting Started

There are few key concepts to know before getting started with the HPC.

  • All jobs, data, and programs must be ran from the /workspace/[user] or /workspace2/[user] directory on either lunchbox.stat.wisc.edu or jetstar.stat.wisc.edu. This is because those locations are visible by all nodes on the cluster. Create your user directory with mkdir -p /workspace/[username] under /workspace using your statistics username.
  • You must use software located in /workspace/software or /s. Some software in /usr/bin will also work, but try to use software from these first two locations when possible.
  • If using R, your path to R packages must be accesible by each node on the cluster. Therefore, all packages and special libraries needed by your job must be located in /workspace/[user]/[someDirectory] or request that the lab install packages for you in the global root area which is also universally accessible by all nodes on the cluster. See the section on R libraries for more information on how to set your R library path.
  • Always submit small/simple jobs to the cluster first before expanding to larger, longer running jobs.
  • Data stored in /workspace and /workspace2 is not backed up and is periodically cleaned out at the beginning of semesters. Move all important data to your AFS home directory for backup or move to an external drive. There is 8 terabytes of storage in both /workspace and /workspace2 and it is shared among all users. If you have need for large data storage or increased security please email the lab.

The steps in which you’ll proceed to start using the HPC cluster are:

1. Email the lab or stop into the lab to request an account. You must provide your Statistics username and be part of the Statistics program at UW-Madison.
2. Log into a submit node and transfer any data to your working directory which can either be /worksace or /workspace2.
3. Test your job in an interactive session. Determine how many cores/threads your job requires.
4. Submit a simple job to the cluster to know that your submit script is working, then you may expand it to more cores if necessary, and longer run times.
5. Consult with the lab for help if problems persist.
6. Remove any data generated by you in your working directory when you are finished. Data in /workspace and /worksapce2 are not backed up and this space is intended for running jobs only. Long term backup or storage of your work should be in your AFS user space or elsewhere.