How to Get an Interactive Session on an HPC Compute Node

A Slurm login node is a shared front door, not a workbench. Every person on the cluster uses it at the same time to edit files, submit jobs, and check queue status — and its CPU, memory, and process table are shared across all of them. When you run a memory-heavy Python session, compile code with make -j, or leave a Jupyter kernel idle-but-resident there, you are taking cycles away from everyone else logged in at that moment. Most clusters enforce this with automated reapers — cgroup-based resource caps, or a monitoring daemon like Arbiter2 — that throttle or kill processes exceeding the login node’s usage policy without warning. The fix isn’t to work more carefully on the login node; it’s to stop working there at all. Slurm’s salloc and srun --pty commands get you a real interactive shell on an actual compute node, allocated to you alone, in about the same time it takes to type the command.

Why the login node is off-limits for real work

A login node exists to do exactly three things: let you edit and manage files, let you submit and monitor jobs, and run the lightweight client tools (squeue, sacct, module commands) that talk to the scheduler. It is not sized for computation, because it is one machine shared by every user of the cluster simultaneously — sometimes hundreds of people at once. A single runaway script, a debugger session left attached to a large process, or an interactive R/Python session loading a multi-gigabyte dataset can measurably degrade the login node for everyone else trying to do the same lightweight tasks.

This is why HPC centers publish explicit login-node usage policies and back them with automated enforcement. The exact mechanism varies by site — some use cgroup memory/CPU limits scoped to the login node, others run a watchdog process like Arbiter2 that detects and kills offending processes — but the outcome is the same everywhere: work that belongs on a compute node, run on the login node, gets killed, and it can happen mid-debug-session with no save prompt. Requesting a compute node through Slurm isn’t a courtesy; it’s the only place your interactive work is actually safe to run at real intensity.

The two-step pattern: salloc, then srun --pty

The canonical way to get an interactive shell on a compute node is two commands, not one. First, request an allocation with salloc:

salloc --nodes=1 --ntasks=1 --cpus-per-task=4 --mem=16G --time=02:00:00 --partition=interactive

This reserves the resources you asked for and, by default, drops you into a new shell that has the allocation’s environment variables (SLURM_JOB_ID, SLURM_NODELIST, and so on) set. On many clusters that shell is still technically running on the node you launched salloc from — usually the login node — because salloc‘s job is to obtain the allocation, not to relocate your terminal. You still need srun to actually launch a process onto the node Slurm assigned you:

srun --pty bash

Run inside the shell salloc gave you, this launches an interactive bash job step that executes on the allocated compute node, with a pseudo-terminal (--pty) attached so it behaves like a normal, directly-connected shell — you can run commands, see output live, and Ctrl-C a hung process exactly as you would locally. When you exit that shell, you’re back in the outer salloc shell with the allocation still held; exiting that one (or letting --time run out) releases it. See the CASRAI srun vs. sbatch vs. salloc comparison for how these three commands relate to each other more generally.

The one-line shortcut: srun --pty alone

If you don’t need to run multiple separate commands against the same allocation, you can skip salloc entirely. srun creates its own resource allocation automatically if you invoke it outside of one, so a single line gets you the same result:

srun --nodes=1 --ntasks=1 --cpus-per-task=4 --mem=16G --time=02:00:00 --partition=interactive --pty bash

This is the version most people reach for in practice: request resources and drop into a shell on the compute node in one step. Reserve the two-step salloc pattern for cases where you genuinely want to run several separate srun steps — for example, one step to profile a script and a second, differently-resourced step to test a fix — against the same held allocation.

Options you’ll actually set

Both commands accept the same core resource flags. None of these have a universal default that suits every job, so set them deliberately:

• --time=HH:MM:SS — wall-clock limit for the session. Interactive sessions should be short; a long --time on a busy partition means a longer wait in queue.
• --nodes / --ntasks — how many nodes and tasks to reserve. For a single interactive debugging shell this is almost always --nodes=1 --ntasks=1.
• --cpus-per-task — CPU cores available to that task, relevant if your script itself threads or forks.
• --mem (or --mem-per-cpu) — real memory to reserve. Slurm enforces this as a hard cap; exceeding it gets your job step killed with an out-of-memory error, the same way it would in a batch job.
• --partition — many clusters run a dedicated short-wait queue for interactive work (commonly named interactive, debug, or similar) separate from the general batch partitions. Check your cluster’s documentation or run sinfo to see what’s available.
• --gres=gpu:1 — on GPU partitions, request a generic resource the same way a batch job would; an interactive session gets no GPU access without it.

Every one of these is a real salloc/srun option, documented in Slurm’s own reference pages at slurm.schedmd.com/salloc.html and slurm.schedmd.com/srun.html. Confirm exact partition names and defaults with your own site — they are set locally and vary between clusters.

Cluster-specific wrapper scripts

Because the full salloc/srun --pty invocation is verbose and the right flags differ by site, many HPC centers wrap it in a short local script — often named something like interactive or sinteractive — that fills in sensible defaults (partition, time limit, memory) for that specific cluster. If your center provides one, prefer it: it encodes local policy (queue names, default limits, accounting flags) that a generic salloc command from a tutorial won’t know about. Check your cluster’s user documentation or run which sinteractive / ask your HPC support team before assuming the plain Slurm commands above are the whole story locally.

When an interactive session is the right tool

Interactive sessions exist for the parts of computational work that don’t tolerate a submit-and-wait cycle:

• Debugging a script interactively. Attaching pdb, ipdb, or a compiled-language debugger to a process, stepping through code, and inspecting variables live requires a persistent, responsive shell — something a batch job’s fire-and-forget execution model doesn’t give you.
• Testing before submitting a full job. Before committing to a large sbatch run that might occupy hundreds of core-hours, run a small-scale version interactively first: confirm the environment loads correctly, the data path resolves, and the first few iterations produce sane output. Catching a typo’d file path in an interactive shell costs a minute; catching it after an 8-hour batch job fails costs the job’s entire wall-clock allocation.
• Exploratory data work. Poking at a new dataset, checking dimensions, testing a parsing approach — the same reasons you’d want a live REPL on your own laptop apply on the cluster, just with access to data or hardware too large to move locally.
• Building or compiling with real resources. A build that needs more memory or CPU than the login node’s shared allowance permits (a large C++ project with parallel make -j, for instance) is exactly the kind of load that should never touch the login node in the first place.

If you’re setting up the software environment those sessions depend on, see CASRAI’s guides on sharing a reproducible conda environment and using conda, pip, and mamba together, or the dictionary entry on container images (Docker/Singularity/Apptainer) if your workflow is containerized.

When to stop working interactively and submit a batch job

An interactive session is for iteration, not for the actual run. Once your script works correctly at small scale, move the real computation to a batch submission with sbatch — it queues independently of your terminal staying open, gets fair scheduling alongside everyone else’s batch work, and doesn’t hold an interactive allocation idle while it runs unattended for hours. CASRAI’s guide on how to write an sbatch job script covers the anatomy of that submission. If you need to run the same job across many inputs, Slurm job arrays handle that without a submit loop, and if your workload is naturally parallel across independent units of work, see embarrassingly parallel jobs for how to recognize and structure that pattern.

Ending the session cleanly

Type exit (or Ctrl-D) to leave the srun --pty bash shell first — this ends that job step but leaves the outer allocation held if you started with salloc. Exit that outer shell the same way to release the allocation, or run scancel with the job ID shown when the allocation was granted if you need to release it from another terminal. Don’t rely on your --time limit to clean up after you; an allocation held open and idle is resource that could be running someone else’s job, including your own next one.

Frequently asked questions

Do I need salloc at all, or can I just run srun –pty bash?

srun --pty bash on its own works fine and is what most people use day to day — srun creates its own allocation automatically when run outside of one. Use the explicit two-step salloc pattern only when you want to run more than one srun step against the same held allocation.

Why did my interactive session get killed on the login node?

You likely ran the compute directly on the login node instead of requesting an allocation first. Most clusters run automated enforcement (cgroup limits or a watchdog like Arbiter2) that kills or throttles processes exceeding login-node CPU/memory policy, with no warning beyond the process dying. Request a compute node with salloc or srun --pty before running anything resource-intensive.

How long can an interactive session stay open?

Whatever --time you request, subject to your partition’s maximum — interactive/debug partitions typically cap this lower than general batch partitions (often an hour or a few hours) precisely because the resource is meant to turn over quickly. Check your site’s partition limits with sinfo or your local documentation.

Can I run GPU code in an interactive session?

Yes, if you request a GPU partition and the generic resource, e.g. --partition=gpu --gres=gpu:1, the same way you would in a batch script. Without an explicit --gres request, an interactive shell has no GPU access even on a GPU-equipped node.

What happens to my job if the connection drops?

The allocation and any running srun step are tied to your terminal session, not your SSH connection’s continuous liveness in every configuration — but a dropped connection commonly does end the interactive shell. Running your SSH session inside tmux or screen on the login node, then salloc/srun from inside that multiplexer, is the standard way to survive a dropped connection without losing the session.

Is an interactive session billed the same as a batch job?

On clusters with allocation accounting, yes — interactive time consumes the same core-hour or service-unit budget as a batch job of equivalent size and duration. An idle interactive shell holding resources it isn’t using still counts against your allocation, which is the main reason to close a session as soon as you’re done with it rather than leaving it open in a spare terminal.