Homework 0

September 8th, 2019

Homework 0 is a simple introduction to the HPC and running parallel codes. All homeworks should contain the scripts that were executed and a writeup explaining what you did and what the solution looked like. Email a zip folder with these contents to crackauc@mit.edu.

For your writeup, you may want to try Weave.jl for mixing code with output webpages or PDFs. Other systems to try are Literate.jl, Lyx, or plain ol' Latex (Overleaf is a good service to try if you haven't used it already!). I would highly recommend not writing the output in a word document.

Problem 1

Get access to the Supercloud HPC and run the Parallel Word Count example. Share the output in your writeup.

Problem 2

Setup a parallel code to run on the HPC with multiple nodes from scratch. For this, let's generate N=50 Brownian motions using a parallel map (pmap). A Brownian motion can be easily made through the following function:

make_brownian(dt,n) = cumsum([0;sqrt(dt).*randn(n+1)])

make_brownian (generic function with 1 method)


Create a .jl file which uses Distributed, addprocs, @everywhere, ClusterManagers, pmap, and the lines

addprocs(SlurmManager(parse(Int,ENV["SLURM_NTASKS"])-1))


to create 50 Brownian motions with dt=0.1 and n=100 using various compute nodes across the cluster. Demonstrate that this setup is actually using multiple nodes (hint: use the hostname command line command). Write a .sh batch script to run this process in parallel. In your writeup, describe how you know that your commands are being run on multiple nodes. (The total number of cores you request doesn't matter, as long as you receive more than 1 node!)