HPC 201: Advanced High Performance Computing Workshop

Overview

This course will cover some more advanced topics in cluster computing on the U-M Flux Cluster. Topics to be covered include a review of common parallel programming models and basic use of Flux; dependent and array scheduling; advanced troubleshooting and analysis using checkjob, qstat, and other tools; use of common scientific applications including Python, MATLAB, and R in parallel environments; parallel debugging and profiling of C and Fortran code, including logging, gdb (line-oriented debugging), ddt (GUI-based debugging) and map (GUI-based profiling) of MPI and OpenMP programs; and an introduction to using GPUs. We will issue you a temporary allocation to use for the course, or you can use your existing Flux allocations, if any.

Prerequisites

HPC 101 or equivalent. This course assumes familiarity with the Linux command line, text editing on Linux, and a basic understanding of Flux including how to submit and track jobs.

Instructors

Dr. Charles J Antonelli
Advocacy and Research Support Group
LSA Information Technology

Charles is a High Performance Computing Consultant in the Advocacy and Research Support Group of LSAIT at the University of Michigan, where he is responsible for high performance computing support and education, and is an advocate to the Departments of History and Communications. Prior to this, he built a parallel data ingestion component of a novel earth science data assimilation system, a secure packet vault, and worked on the No. 5 ESS Switch at Bell Labs in the 80s. He has taught courses in operating systems, distributed file systems, C++ programming, security, and database application design.

Seth Meyer
Advocacy and Research Support Group
LSA Information Technology

Mark Champe
Advocacy and Research Support Group
LSA Information Technology

 

 

Materials

Course Preparation

In order to use the Flux cluster successfully, you must have a Flux user account, a Flux allocation, and an MToken. The user account allows you to log in to the cluster, create, compile, and test applications, and prepare Flux jobs for submission. The allocation allows you to submit those jobs, executing those applications in parallel on the cluster and charging their resource use against the allocation. A token is required to authenticate you to the cluster.

Flux user account

A single Flux user account can be used to prepare and submit jobs using various allocations. If you already already possess a user account, you can use it for this course, you can skip to “Flux allocation” below. If not, please fill out this form to obtain one. A user account is free to members of the University community.

On the web form you’ll be asked to fill out, you should enter your own project and software information. If you don’t have this, good defaults include: for “Project(s) for which you will use the CAC computers,” enter “HPC 201″; for “What software will you be using,” enter “gcc, icc, ifort.”

Please note that obtaining an account requires human processing, so be sure to do this at least three business days before class begins.

Flux allocation

We’ll issue you a temporary allocation so you can run jobs on the cluster during the course. If you already have an existing Flux allocation, you can use that as well, if you like.

MToken

An MToken (or a Software Token) is required to login to the cluster. When logging in, you will need to give both the passcode it displays at time of login as well as your Kerberos password in order to authenticate. If you already possess an MToken, you can ignore the rest of this paragraph. Otherwise, you should do one of the following:

  • Obtain a token by visiting the LSAIT Help Desk in Suite 1112 LSA Building, 500 S State St. The Help Desk will issue you an MToken and instructions for activiating it. Your simplest course of action is to activate your MToken before you leave the Help Desk area. If you cannot do so, please activate your MToken before you come to class.
  • Request a token from the LSAIT Help Desk by sending an email to lsa-csg@umich.edu or by calling 734 936 3279, and we will send you an MToken through the campus mail. Please make your request no later than four business days before class, and please activate your MToken before you come to class.
  • Show up for class at 12:30 PM, and we will issue you an MToken and instructions for activating it. Note you will need to be able to send and receive email at that time to complete the activation process.

Laptop Preparation

You do not need to bring your own laptop to class. The classroom contains Mac OS X computers, which require your uniqname and Kerberos password to login, that have all necessary software pre-loaded.

If you want to use a laptop for the course, you’ll minimize in-class distractions if you prepare your laptop as follows well before class begins:

  • You will need an ssh client to connect to the Flux cluster. Here are a couple of choices:
  • Optionally, you can use an X server on your laptop to open GUI displays from the Flux cluster. Mac OS X and Linux platforms have this built-in. If you have a Windows platform, consider downloading and installing Xming (http://sourceforge.net/projects/xming/). Visit the “Project home” link at this URL for additional documentation.

More help

Please use our class mailing list hpc201course@umich.edu for questions, comments, or to seek further assistance.