a reasonably popular job schedule that is deployed on a number of computing clusters is slurm. Slurm is just like any other job schedule, like grid engine or others like that, that allows jobs to be submitted to it, manages the queue, and then pushes those jobs onto the compute clusters for operation. Now, there are a few commands that allow you to request jobs: `snes batch`, `srun`, `salec`. These are commands that allow you to request a job, allocate, and put a job on the job queue. `scancel`, as the name probably sounds, is the command that lets you cancel a job off of the queue. Then there's a handful of commands that allow you to look at the various states of the system. `squeue`, as it suggests, is the tool that queries the slurm queue, the job queue itself. `sinfo` queries information about the compute cluster, and `saccount` will query information about jobs or users, and other aspects of the environment. These are the three, these are the handful of commands.

if you can, if you can, you know, navigate your way around, around, you know, s batch, s cancel, and sq um, and maybe, s info, you've got pretty much everything you need to, to work through. This is there's a few other commands but these are the big ones that that allow you to interact and and work with the system. So how do you go about now submitting a job to, to the to a cluster that uses slurm, for example? Well, s batch, s run, and s alec all have parameters and there's a common set of parameters that they all use that define the jobs. So these three commands allow you to submit jobs into the queue for the slum scheduler to manage and when you're submitting a job, there are some defaults but generally the defaults aren't what you want. You want to specify additional parameters that that describe to slurm what type of job you're you're requesting with what type of resources. So some one thing that could be important to know, for example, is how many cpus per task and we're only

going to talk about a single task a job for now. So in this case, how many CPUs are you going to be requesting? That's specified with the dash lowercase c flag. So you would say something like s batch dash c, and then you would put the number in of the number of CPUs you're asking for. If you only want a single CPU per task, right per your job in this case, if there's only one task, then you're asking for c1. If you need 10 CPUs, right, because you want to run something that's going to use 10 CPUs concurrently at the same time, then you would say dash c10. You just, you ask, you're telling the Slurm how many CPUs you need for your job on a per task basis. Then, if you, you can also specify or you need to specify the memory. So how much memory per CPU are you asking for? So if you're asking for say a gigabyte of memory per CPU, then you would put in here, you know, 1g.

and this is this is pretty intelligent. You can it actually knows the suffixes so you can you know say 100 megabytes, right? I think you actually just do 100 m. I think it understands that. Or you can do something like 4g and you ask for four gigs of memory. And these are on a per cpu basis. So if you asked for dash c3, right? Three cpus, but then you specify dash dash mem per cpu and you said, you know, four gigs, then your job in total is actually going to be allocating 12 gigabytes of memory because it's this is a per cpu basis. And then the last parameter that's, you know, relatively important specify is time. How long do you need the resources of the cpu and the memory that you're asking for? How long do you need them for? That you have to know how long the job that you're going to that you're submitting is going to run. Now you might say, well, I have no idea how long this is going to run. Well, you can just try and guess some things.

and then you can go back and look to see how long it actually took, or you can underestimate it, maybe, and find that it didn't run. And then you have to, you know, jump it up a bit more. But this is something that you'll have to sort of get some intuition on to get a sense of how long it takes for a particular for a particular job to execute. Now you could say, well, why don't I just ask for as much time as possible? Well, in that case, you might be sitting around on the job queue for a very long time. It actually is in your interest to ask for as few resources as possible, both in CPUs, memory, and time, because that makes it much easier for the job scheduler to slot you in when there's an opening, right? If there's a much bigger job sitting around, then that's much harder to position into the queue or position to find resources for than a much smaller job. So if you don't want your job to be sitting around in the queue for quite some time.

it's best. It's in your best interest to trim these to just what you actually need. And time is specified. You know in units as well. So it's done with. It's done with um. It's done with some simple syntax. So you would say something like, you know, 10 colon colon 0 if you wanted 10 minutes, right? Or if you had, you know, colon colon 0 here, this would be 10 hours of time. And if you wanted like 3 days, it would be dash zero colon zero colon zero. This is three days of time being allocated. It's a pretty simple syntax. So just to round it out completely, let me show you a full sbatch example. And this, for example, is an example call that will submit a batch process to the serm queue. So in this particular example, you'll notice I'm asking for two CPUs. I'm asking for two gigs per CPU. So my job will take up four total gigabytes of space of memory. And I'm asking that for five hours.

all right because this would be seconds, minutes, and hours. I'm introducing a new parameter here, j, which I hadn't told you about. That's just the job name. I'm labeling it in the job queue so when you sq it, you can actually identify quickly what you've submitted. I'm calling this job the crunchy name. That's what I'm naming it. And then this part I hadn't shown you before, but this --wrap is an instruction to sbatch on actually the call to make, meaning what is it actually going to be submitting when it gets allocated, what command is it going to be running. And that's what this is. It's going to be running the python crunchy.py call. So you can type in anything you want here and it will execute. You can point it to a script and it can execute that. There's another, there's also other ways of writing these sbatch calls with command files, but this is a simple command line way of doing it. So this right is for sbatch if you wanted.

s run takes all these same flags, but it doesn't allow you to do the wrap command. Instead, sron is a blocking call where it actually will sit here and wait. You won't have access to the terminal until that s s1 is complete because you've specified at the end what you're running at the end of this. So there's no wrap call. It's actually all in line. And so python would appear in here, and this would wait, try to allocate resources. Once it allocates resources, it would then actively run this command, and your command line would just be waiting there until it was done, and you'd see all the output. S batch instead submits the job queue and returns control to you so that you can go about doing other things while you're waiting. And then just to be complete, s alak is all of the same parameters, only there's no wrap command here. It just lets you declare resources, and it will allocate them whenever it gets a chance, and then it will do anything else. It'll just be a job that's out there.

without anything running. And then you can jump into the compute into the system and work on it in an interactive sense if you want it or submit it manually. S Alex just allocates resources. It doesn't actually run anything. If you want to allocate resources then run something, you have to use either sbatch, which is the one that will submit things in the background, or s run, which will submit things in the foreground and actively execute them while you watch. But it could take, you know, they both, they both take the same amount of time. It's just whether or not you have control over your terminal in the meantime.

Slurm Job Scheduler Basics

Slurm Job Scheduler Basics