Advanced usage

This page describes some less-than-obvious options that can be used with crtools.

Evasive devices

Sometimes an application opens a device file and then somehow the path by which it was opened becomes inaccessible (e.g. overmounted or unlinked). In that case crtools cannot easily dump and restore such a process. If you consider that the path doesn't really matter when dumping your apps state you can tell crtools that a device file can be opened by any name, even if the original one is no longer accessible. The option for that is --evasive-devices

External UNIX sockets

Consider an application opens a datagram UNIX socket and connects it to some address. If you will try to dump such app and the server socket for some reason will not be taken in the dumped state (e.g. -- a task holding it is not dumped) the dump will fail. You can override this behavior by allowing crtools to disconnect the client after dump and re-connecting it back on restore by the server socket path using the --ext-unix-sk option.

Link unlinked files back

When an app opens a file and then unlinks it we take this file with us into image. However, we do so only if the file is really unlinked, i.e. its n_link is zero. Otherwise we cannot take file with us and should somehow open the inode. To do so we should create a temporary hardlink on it during dump, open it on restore and unlink again. The --link-remap option allows crtools to create this temporary hard links on FS.

TCP connections

When dumping and restoring an application having an opened tcp connection you should use the --tcp-established option. When this option is in use crtools will leave the connection(s) locked after dump and will require it(them) to be still locked before restore.

Per-task logging on restore

By default crtools puts all logs into one log file specified by -o/--log-file option. If you want to split restoration logs on per-pid basis you can use the --log-pid option. The ${pid} task's logs will appear in a ${log-file}.pid

Knowing a new pid of restored task

When you restore a process tree with -n/--namespaces pid option the pid of a new task will be generated by the system (since the pids sitting in image files are treated as virtual, i.e. seen in namespace ones). To find this pid out you can use the --pidfile <file> option to make crtools put the new pid into a pidfile.

Restoring a veth pair of devices

When you restore a net namespace with a veth device end in it, crtools will create the other end of the pair in the net namespace you launch crtools from. By default it's name will be the one generated by veth kernel driver. Not to scan through all the net devices trying to fine one you can use the --veth-pair <ns-dev>:<host-dev> option. When used a device named <ns-dev> in a newly restored namespace will be linked with the <host-dev> one in the crtools namespace.

Action scripts

The crtools can call your hooks on various stages of dumping/restoring. These hooks are added with the --action-script shell-code-to-execute option. When called, the CRTOOLS_SCRIPT_ACTION environment is set to a value determining which type of action is performed. See Action scripts for more details.

Shell jobs C/R

When you run some app directly from shell (e.g. -- just launch top) it inherits session ID from bash and uses pty whose other end sits somewhere outside this shell session (e.g. in sshd or xterm). Strictly speaking this app cannot be checkpointed (since session leader, which is bash, is left outside the image, and so is the pty master). Nor it can be restored easily. But sometimes it might make sense to migrate such app by moving it to other session and attaching to different pty slave "on the fly". For such cases the --shell-job option should be used on both stages -- dump and restore. On dump it will allow crtools to ignore the "leaked" session leader and pty master, on restore it will tell crtools to change session, group and attach to existing pty slave.

File locks C/R

Some app may use file locks for synchronization. Generally they will use flock or posix file locks, which were achieved by flock or fcntl system calls. For dump/restore, it is hard to be handled perfectly, because we can't guarantee all potential users are dumped for a specific file lock. Right now, we assume that all file lock users are taken into dump, and we should use the --file-locks option on both dump and restore stages if our app may use any file locks. Remember that file locks dump/restore is only safe for container dumping in theory.