This workshop provides researchers in the humanities, arts, and sciences an opportunity to become familiar with containerization tools, which are in wide use in the research computing world. The workshop will introduce the tools and provide hands-on experience. Researchers with software they want to package -- or "containerize" -- for use on the cluster or elsewhere will be able to do so. We will introduce Gitlab-CI (https://about.gitlab.com/features/gitlab-ci-cd/) and Singularity ( https://sylabs.io) and tools that OIT programmers have built to automate the production of "cluster-ready" Singularity containers. Who should take part? Users of the Duke Compute Cluster; researchers who are using software with specific and perhaps exotic library dependencies; researchers who are using computers at other locations, including XSEDE supercomputers, Open Science Grid (OSG), or computers shared with colleagues at other institutions, researchers who want to increase the likelihood that their computational methods are transportable and reproducible. Time will be available for participants to ask questions about building their own containers.
This class will be hosted via Zoom. You will receive an email with the Zoom link a day before the class. Attendance will be taken and the session will be recorded.
- Singularity overview
- Create a singularity image
- Using Git CI to update and manage containers for multiple members of your lab
- Execution of Singularity Images
- Use of Singularity in the Duke Compute Cluster
- Example definition files
- Researchers will be able to package Linux compute environments (e.g. Jupyterlab, Caffe) and be able to run those containers on the DCC or at National Labs like Xsede or OSG.
- For researchers familiar with Docker, you will be able to port your Docker containers to Singularity.
Create an account on Duke's internal Gitlab service (https://gitlab.oit.duke.edu/). Duke's Gitlab uses Shibboleth authentication and is freely available to anyone with a NetID. Reserve a Linux VM from https://vcm.duke.edu or have privileged access to an existing Linux server.
People taking part should be familiar with using Git and comfortable with the Linux command line is required. If you need help with the Cluster Computing, Git or Linux command line check out our additional co-lab classes: Linux & the Bash Shell, Backing Up and Sharing Code: Git, High Performance Computing and the Duke Compute Cluster
This class is part of our Research Computing TrackSee the whole track
High Performance Computing and the Duke Compute Cluster
The Duke Compute Cluster (DCC) is a general purpose, high performance, Linux computing cluster with software used for a broad array of scientific projects. This workshop will provide Duke University researchers with a brief overview of Research Computing resources including the Duke Compute Cluster.Learn More