Moving Files With scp and sftp

Goal

With scp and sftp you upload and download files, preserve timestamps, and confirm integrity with checksums.

Why it matters

A transfer being "finished" and it having "arrived correctly" are different things. In practice you always confirm with checksums. And the fact that scp's port option is uppercase -P looks trivial, but because in ssh lowercase is the port, people really do get it wrong often. Lowercase -p is preserve timestamps, so it quietly does something different and moves on. sftp batch mode is still the standard for automating file exchange with partner companies.

In this lab, think of 127.0.0.1:2222 as a different server. You use the /root/.ssh/id_labhub key you made in the previous lab as it is.

Steps

Create the /root/xfer/src directory and create three files inside it: a.txt (at least 1KB), b.txt (at least 1KB), and c.bin (at least 10KB).
Create the remote directory /root/xfer/dst/ and upload a.txt there.
Recursively copy the whole /root/xfer/src directory under the remote /root/xfer/tree/. As a result, /root/xfer/tree/src/b.txt must exist.
Download the remote /root/xfer/dst/a.txt to the local /root/xfer/back/.
Write sftp batch commands in /root/xfer/batch.txt and run it. The batch must upload /root/xfer/src/c.bin to the remote /root/xfer/dst/. Save the execution log to /root/xfer/sftp.log.
Copy b.txt to the remote /root/xfer/dst/ with timestamps preserved, and write the mtime (in seconds) of the original and the copy, one per line, in /root/xfer/mtime.txt. The two values must be the same.
Make a sha256 manifest of the three files in /root/xfer/src, verify that the hashes of the files that arrived in /root/xfer/dst and /root/xfer/tree/src match, and save the result to /root/xfer/verify.txt.
Make /root/xfer/notes.txt with the following 3 lines. SCP_PORT_FLAG=-P / SSH_PORT_FLAG=-p / SCP_PRESERVE_FLAG=-p

Notes

The form is scp -P 2222 -i /root/.ssh/id_labhub <원본> root@127.0.0.1:<대상> (the placeholders are the source and the destination).
To create a directory remotely, just run the command remotely, like ssh lab2 mkdir -p /root/xfer/dst.
An example sftp batch file is cd /root/xfer/dst followed on the next line by put /root/xfer/src/c.bin.
Common mistake 1: using lowercase -p in step 2 so the port goes to 22. The connection itself fails.
Common mistake 2: in step 3, the directory depth changes depending on whether the destination path ends in a slash. Check the required final path.

Preparing the files to transfer

Create the /root/xfer/src directory and create three files inside it: a.txt (at least 1KB), b.txt (at least 1KB), and c.bin (at least 10KB).

Three files of different sizes are enough. You may make them with head or dd, or fill them with base64.

Uploading a file

Create the remote directory /root/xfer/dst/ and upload a.txt there.

Specifying the port is the uppercase option. Be careful, because the lowercase one means something completely different.

Uploading a whole directory

Recursively copy the whole /root/xfer/src directory under the remote /root/xfer/tree/. As a result, /root/xfer/tree/src/b.txt must exist.

To send a directory you need the recursive option. If the destination path does not exist, create it beforehand.

Downloading

Download the remote /root/xfer/dst/a.txt to the local /root/xfer/back/.

You write the remote path first and the local path after. The remote notation is in the user@host:path format.

Running an sftp batch

Write sftp batch commands in /root/xfer/batch.txt and run it. The batch must upload /root/xfer/src/c.bin to the remote /root/xfer/dst/. Save the execution log to /root/xfer/sftp.log.

sftp has an option that reads a list of commands from a file. In the batch file you write one command per line.

Preserving timestamps

Copy b.txt to the remote /root/xfer/dst/ with timestamps preserved, and write the mtime (in seconds) of the original and the copy, one per line, in /root/xfer/mtime.txt. The two values must be the same.

The preserve option is lowercase. Check by comparing the mtime with stat.

Integrity verification

Make a sha256 manifest of the three files in /root/xfer/src, verify that the hashes of the files that arrived in /root/xfer/dst and /root/xfer/tree/src match, and save the result to /root/xfer/verify.txt.

Make the manifest with sha256sum and check with -c. It is convenient to match the paths so that the original and the copy have the same name.

Options summary note

Make /root/xfer/notes.txt with the following 3 lines. SCP_PORT_FLAG=-P / SSH_PORT_FLAG=-p / SCP_PRESERVE_FLAG=-p

The key point is that the case of the port option differs between scp and ssh. Write the values with the exact characters.