Direct comparison
rsync vs scp for HPC Data Transfer
rsync resends only changed bytes and can resume; scp always copies the whole file. Compare syntax, speed, and when each fits HPC data transfers.
Written and maintained by CASRAI Editorial Board
Last updated
Ask CASRAI · included with Regulatory Radar
Ask about rsync vs scp for HPC Data Transfer
Ask CASRAI answers research-administration questions and cites the passages behind every claim — and says so when the corpus does not cover something, instead of guessing. It comes with a Regulatory Radar subscription at $29 a month, alongside the daily digest of regulatory changes and the dashboard of what changed.
150 questions a day, on this site, over the API, or inside your own tools through the CASRAI MCP server.
Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.
How do rsync, scp compare side by side?
The table below compares rsync, scp across 13 procurement-relevant dimensions, from core transfer mechanism through best-suited scenario.
Side-by-side comparison
| Dimension | rsync | scp |
|---|---|---|
| Core transfer mechanism | Delta-transfer algorithm: compares source and destination files (by size and modification time, or by checksum with -c) and transmits only the differing blocks. | Full copy every run. scp has no comparison step — it reads the source file and writes the entire thing to the destination, regardless of what is already there. |
| Repeating a transfer of the same directory | Second and later runs are fast: unchanged files are skipped entirely, and changed files transfer only their altered blocks. | Every run re-copies every file in full, even files that have not changed since the last transfer. |
| Resuming an interrupted transfer | Yes, with --partial (or the -P shortcut, which combines --partial and --progress). A dropped connection leaves the partial file in place and re-running the same command picks up from where it left off. | No native resume. The OpenBSD scp manual documents no partial-transfer or resume capability — an interrupted copy must be restarted from the beginning. |
| Underlying transport | Runs over SSH by default on HPC systems (rsync -e ssh, or simply the ssh-style host:path syntax), or its own rsync:// daemon protocol for public mirrors. | Runs over SSH exclusively — it is a thin wrapper around the SSH protocol’s file-copy capability. |
| Typical command: local directory to a cluster | rsync -avzP ./local_data/ [email protected]:/scratch/user/project/ | scp -r -C ./local_data [email protected]:/scratch/user/project/ |
| Trailing-slash behavior on the source path | Matters: a trailing slash on the source (local_data/) copies the directory’s contents into the destination; omitting it copies the directory itself as a subfolder. | No equivalent distinction — scp -r always copies the named directory itself into the destination. |
| Compression in transit | Built in via -z (compress file data during the transfer), independent of any SSH-level compression. | -C enables compression by passing the flag through to the underlying ssh(1) connection. |
| Preserving permissions, ownership, timestamps | Archive mode -a (equivalent to -rlptgoD) preserves permissions, symlinks, timestamps, group and owner, and device files in one flag. | -p preserves modification time, access time, and file mode bits; ownership and symlink handling are more limited than rsync’s archive mode. |
| Mirroring — removing destination files no longer at the source | Yes, with --delete. Makes the destination an exact mirror of the source, which is useful for keeping a working copy in sync with a canonical dataset. | Not supported. scp only adds and overwrites; it never removes files from the destination. |
| Overhead on a single small one-off file | Real but usually trivial overhead: rsync still has to check whether the destination file exists and compare metadata before deciding whether to send data. | None — scp connects and copies with no comparison step, which for a single small file already destined to be fully transferred is simply less work done. |
| Behavior on a large, mostly-unchanged dataset | This is rsync’s core advantage: re-syncing a multi-gigabyte directory after a small edit transfers only the changed blocks, often in seconds instead of the minutes or hours a full re-copy would take. | Re-copies the entire dataset every time, at full transfer time, regardless of how little actually changed since the last copy. |
| Availability on HPC systems | Present on essentially every modern Linux HPC login node; if genuinely missing, most centers make it available via an environment module. | Ships with OpenSSH, which every HPC login node already runs for interactive access — no separate installation is ever needed. |
| Best-suited scenario | Large datasets, repeated syncs during active analysis, mirroring a working directory, or any transfer that might need to resume after a dropped connection. | A single small file or a one-off transfer that will only ever run once — where rsync’s file-comparison overhead has nothing to amortize against. |
Common questions
Common questions about rsync vs scp
Is rsync always faster than scp?
+
No. rsync is faster on repeat transfers of large, mostly-unchanged data because it skips or partially skips files. On a single one-off transfer of a small file, rsync still spends time checking file state before copying, so scp can be marginally quicker simply because it skips that check.
Can I resume an interrupted scp transfer?
+
No. scp has no partial-transfer or resume mechanism — an interrupted copy has to be restarted from the beginning. rsync with --partial (or the -P shortcut) is the tool to use when a transfer might be interrupted, such as a large dataset moved over an unreliable connection.
Does rsync verify that the transferred data is correct?
+
Yes. rsync’s delta-transfer algorithm is checksum-based — it uses rolling checksums to identify which blocks differ, and by default also compares file size and modification time to decide whether a file needs re-checking at all. Adding -c forces a full checksum comparison on every file instead of relying on size/time.
Do I need to install rsync on the HPC cluster myself?
+
Almost never. rsync is preinstalled on essentially every Linux-based HPC login node alongside standard SSH tooling. If a specific cluster genuinely lacks it, check for an environment module (module avail rsync) before assuming a manual install is needed.
What does the -P flag do in rsync?
+
-P is shorthand for --partial --progress together: it keeps partially transferred files in place if a transfer is interrupted (so a later run can resume) and shows a live progress indicator during the transfer.
Can rsync delete files at the destination that no longer exist at the source?
+
Yes, with the --delete flag, which makes the destination an exact mirror of the source directory. scp has no equivalent — it only ever adds or overwrites files, never removes them.
