Chapter 16 ■ telnet and SSh
312
SFTP: File Transfer Over SSH
Version 2 of the SSH protocol includes a subprotocol called the SSH File Transfer Protocol (SFTP) that lets you walk
the remote directory tree, create and delete directories and files, and copy files back and forth from the local to the
remote machine. The capabilities of SFTP are so complex and complete, in fact, that not only do they support simple
file-copy operations, but they can also power graphical file browsers and can even let the remote filesystem be
mounted locally! (Google the sshfs system for details.)
The SFTP protocol is an incredible boon to those of us who once had to copy files using brittle scripts that tried to
send data across Telnet via a careful escaping of binary data. Instead of making you power up its own sftp command
line each time you want to move files, SSH follows the tradition of RSH by providing an scp command-line tool that
acts just like the traditional cp command, but it lets you prefix any file name with hostname: to indicate that it exists
on the remote machine. This means remote copy commands stay in your command-line history, just like your other
shell commands, rather than being lost to the separate history buffer of a separate command prompt that you have to
invoke and then quit (which was a great annoyance of traditional FTP clients).
Furthermore, the great and crowning achievement of SFTP and the sftp and scp commands is that they not only
support password authentication but also let you copy files using the same public-key mechanism that lets you avoid
typing your password over and over again when running remote commands with the ssh command.
If you briefly scan Chapter 17 on the old FTP system, you will get a good idea of the sorts of operations that
SFTP supports. In fact, most of the SFTP commands have the same names as the local commands you already run to
manipulate files on your Unix shell account, like chmod and mkdir, or have the same names as Unix system calls with
which you might already be familiar through the Python os module, like lstat and unlink. Because these operations
are so familiar, I never need any other support in writing SFTP commands than is provided by the bare paramiko
documentation for the Python SFTP client at http://www.lag.net/paramiko/docs/paramiko.SFTPClient-class.
Here are the main things to remember when doing SFTP:
• The SFTP protocol is stateful, just like FTP and just like your normal shell account. Therefore,
you can either pass all file and directory names as absolute paths that start at the root of the
filesystem or use getcwd() and chdir() to move around the filesystem and then use paths that
are relative to the directory in which you have arrived.• You can open a file using either the file() or open() method (just like Python has a built-in
callable that lives under both names), and you get back a filelike object connected to an SSH
channel that runs independently of your SFTP channel. That is, you can keep issuing SFTP
commands and then move around the filesystem and copy or open further files, and the
original channel will still be connected to its file and ready for reading or writing.• Because each open remote file gets an independent channel, file transfers can happen
asynchronously. You can open many remote files at once and have them all streaming down
to your disk drive or open new files and be sending data the other way. Be careful that you
recognize this, or you might open so many channels at once that each one slows to a crawl.• Finally, keep in mind that no shell expansion is done on any of the file names you pass across
SFTP. If you try using a file name like * or one that has spaces or special characters, they are
simply interpreted as part of the file name. No shell is involved when using SFTP. You are
getting to talk right to the remote filesystem thanks to the support inside the SSH server itself.
This means any support for pattern matching that you want to provide to the user has to be
through fetching the directory contents yourself and then checking their pattern against each
one, using a routine like those provided in fnmatch in the Python Standard Library.Listing 16-7 shows a modest example SFTP session. It does something simple that system administrators might
often need (but, of course, that they could just as easily accomplish with an scp command): it connects to the remote
system and copies message log files out of the /var/log directory, perhaps for scanning or analysis on the local
machine.