Repository navigation
Home
Welcome to the EpiModel Research Lab wiki. It is written for lab members (graduate students, postdocs, and staff) who build and run models with EpiModel and our extension packages, and it may help others who use the same tools. It covers how we develop models on our own computers and how we run them at scale on the Emory RSPH high-performance computing (HPC) cluster.
If you are new, work through the sections in order. Each page starts with who it is for and what to read first. Terms you do not know are in the Glossary.
Some of these take several days, so start them on your first day.
- Emory NetID and Emory VPN access.
- A GitHub account, added to the EpiModel organization. Ask Sam. You need this to see our private repositories, including EpiModelHIV-p.
- Access to the ARTnet data (see ARTnetData).
- An RSPH HPC account (see HPC setup).
- An invitation to the lab Slack workspace.
- Software installed on your computer (see Setting Up Your Computer).
- Questions about our code and workflow (EpiModel, EpiModelHIV,
renv, Spack, Slurm, slurmworkflow): ask in#coding-helpon the lab Slack. - Bugs in one of our packages: open an issue on that package's GitHub repository.
- HPC account, password, VPN, or cluster problems that are not about our code: email RSPH IT at help@sph.emory.edu, starting the subject line with "RSPH HPC cluster:" (see HPC Rules, Storage, and Getting Help).
- Something on this wiki is wrong or unclear: fix it or tell us (see Contributing to This Wiki).
We write and test all code locally and use the HPC only to run it at scale, so your own computer comes first.
- Setting Up Your Computer: R, RStudio, git, GitHub Desktop, compilers, recommended RStudio settings, and a GitHub personal access token.
- Tips for Windows Users
If you are new to network modeling, start here before your first project.
- EpiModel: the platform home page, with a learning pathway for building network models.
- Network Modeling for Epidemics: our short-course materials, with lectures and labs.
- EpiModel documentation: function reference and vignettes on parameters, network objects, and custom attributes.
- EpiModel Gallery: worked examples of custom models.
- The EpiModel paper (Journal of Statistical Software, 2018) and EpiModel in the scientific literature.
Our network models are temporal exponential-family random graph models (TERGMs), fit with netest() and checked with netdx(). For background on ERGMs, TERGMs, and model formulas, see the course chapters on ERGMs, TERGMs, estimation, and parameterization from egocentric data.
- Applied Modeling Project Structure: the project repository, your EpiModelHIV-p branch, and what happens on your computer, on GitHub, and on the HPC.
- Getting Started with EpiModelHIV: create your project from the Template, run a first model, create your EpiModelHIV-p branch, and open your pull request.
- The EpiModelHIV-Template README: the step-by-step setup and the four project chapters (networks, model development, calibration, interventions).
- Using renv for Reproducible R Environments: how each project keeps its own package versions, on your computer and on the HPC.
Our HIV models are parameterized from the ARTnet study. The data are in the private ARTnetData package. To get access, email Sam to request the ARTnet data use agreement; once it is signed, you are given access on GitHub. Details are in the ARTnet README. Without access, installing a project's packages fails with an error about ARTnetData.
EpiModelCOVID projects use the same two-repository pattern, with EpiModelCOVID as the software repository. There is no separate tutorial; ask Sam for a current example project.
- Writing and Debugging EpiModel Code
- Diagnostic of an EpiModel Module
- EpiModelHIV Code Conventions
- Adding Parameters, Attributes, and Modules to EpiModelHIV
Read these in order. Never run R or any computation on the login node.
- Registering for and Logging in to the HPC
-
Setting Up Your HPC Environment: SSH keys, the
ssh sphshortcut, and aliases. - HPC Rules, Storage, and Getting Help: cluster policies, where to put files, quotas, and backups.
- Working with Files on the HPC
- Using Git on the HPC
- Using R via Spack on the HPC
All work on the cluster runs through the Slurm scheduler. Our projects generate their jobs with slurmworkflow (next section), but every HPC user should first learn to run R through plain Slurm: interactive sessions with srun, batch jobs with sbatch, including passing parameters to a script, and checking and canceling jobs with squeue and scancel.
- Running R Interactively with srun: partitions, resource requests, and an interactive session.
-
Running R Batch Jobs with sbatch: job scripts, job arrays, and passing parameters with
--export. The sbatchExample folder in the EpiModeling repository has the complete example. - Monitoring and Managing Slurm Jobs
Project jobs are generated on your computer with slurmworkflow and EpiModelHPC, copied to the cluster, and started there.
- Running Project Workflows on the HPC: build, deploy, monitor, and download a workflow.
- Running Network Models with Checkpoints
Background: the slurmworkflow documentation and the EpiModelHPC article Using slurmworkflow with EpiModel.
- Calibrating a Model: targets, ballpark runs, and automated calibration with swfcalib.
- Making a Restart Point: the saved burn-in that calibration and intervention runs start from, and when to make a new one.
- Programmatically Defining Scenarios
- Processing Intervention Results: outcomes, infections averted, and formatted tables.
The Template's calibration guide lists every file in chapter C, and the swfcalib documentation explains the package.
For the people who maintain the lab's shared HPC setup and this wiki. You do not need these to do your research.
EpiModel Research Lab wiki. Something out of date or unclear? Post in #coding-help on the lab Slack, or fix it yourself: see Contributing to This Wiki.
0. Start here
1. Your computer
2. Learn EpiModel
3. First project
4. Model code
5. HPC
- Access and login
- Environment setup
- Rules, storage, and help
- Files
- Git
- R via Spack
- srun
- sbatch
- Monitoring jobs
- Project workflows
- Checkpoints
6. Calibrate and analyze
7. Publish
8. Admin