Advanced usage

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

This page describes some less-than-obvious options that can be used with criu.

Evasive devices

Sometimes an application opens a device file and then somehow the path by which it was opened becomes inaccessible (e.g. overmounted or unlinked). In that case criu cannot easily dump and restore such a process. If you consider that the path doesn't really matter when dumping your apps state you can tell criu that a device file can be opened by any name, even if the original one is no longer accessible. The option for that is --evasive-devices.

External UNIX sockets

Consider an application opens a datagram UNIX socket and connects it to some address. If you will try to dump such app and the server socket for some reason will not be taken in the dumped state (e.g. -- a task holding it is not dumped) the dump will fail. You can override this behavior by allowing criu to disconnect the client after dump and re-connecting it back on restore by the server socket path using the --ext-unix-sk option.

Link unlinked files back

When an app opens a file and then unlinks it we take this file with us into image. However, we do so only if the file is really unlinked, i.e. its n_link is zero. Otherwise we cannot take file with us and should somehow open the inode. To do so we should create a temporary hardlink on it during dump, open it on restore and unlink again. The --link-remap option allows criu to create this temporary hard links on FS.

TCP connections

When dumping and restoring an application having an opened TCP connection you should use the --tcp-established option. When this option is in use criu will leave the connection(s) locked after dump and will require it(them) to be still locked before restore. Article Simple TCP pair gives an example of using this option.

Per-task logging on restore

By default criu puts all logs into one log file specified by --log-file file option. If you want to split restoration logs on per-PID basis you can use the --log-pid option. The logs of a task having PID of N will appear in a file.N file.

Knowing a new PID of restored task

When you restore a process tree that lives inside a PID namespace, the PID of a new task will be generated by the system (since the PIDs sitting in image files are treated as virtual, i.e. as seen in a PID namespace). To find out the PID, one can use the --pidfile file option to make CRIU put the new PID into file.

Restoring a VETH pair of devices

When you restore a net namespace with a VETH device end in it, CRIU will create the other end of the pair in the net namespace you launch CRIU from. By default its name will be the one generated by the veth kernel driver. Not to scan through all the net devices trying to find one, you can use the --external option. For more details see the article about VETH device.

Action scripts

CRIU can call your hooks on various stages of dumping/restoring. These hooks are added with the --action-script shell-code-to-execute option. When called, the CRTOOLS_SCRIPT_ACTION environment is set to a value determining which type of action is performed.

Shell jobs C/R

When you run some app directly from the shell (e.g. top), it inherits session ID from bash and uses PTY whose other end sits somewhere outside this shell session (e.g. in sshd or xterm). Strictly speaking this app cannot be checkpointed (since session leader, which is bash, is left outside the image, and so is the pty master). Nor it can be restored easily. But sometimes it might make sense to migrate such app by moving it to other session and attaching to different pty slave "on the fly". For such cases the --shell-job option should be used on both dump and restore. On dump, it will allow CRIU to ignore the "leaked" session leader and pty master. On restore, it will tell CRIU to change session, group and attach to existing pty slave.

File locks C/R

Some app may use file locks for synchronization. Generally they will use flock or posix file locks, which were achieved by flock or fcntl system calls. For dump/restore, it is hard to be handled perfectly, because we can't guarantee all potential users are dumped for a specific file lock. Right now, we assume that all file lock users are taken into dump, and we should use the --file-locks option on both dump and restore stages if our app may use any file locks. Remember that file locks dump/restore is only safe for container dumping in theory.

Leave task running after checkpoint

In some scenarios a user might want to leave program in a running state once checkpoint complete. To do that, criu has --leave-running command line option. We strongly encourage NOT to use it, until you understand what you are doing. Leaving task in a running state may lead to inconsistency between task images and external resources such as opened files, TCP connections.

Imagine a task been dumped and then continue execution, for some reason the task close and delete a file which has been checkpointed — any attempt to restore such task will simply fail because CRIU wont be able to open deleted file. Situation become more drastic if TCP connections are checkpointed — during execution the internal state of connections is changed inside kernel and restore then fail as well.

Still, leaving task running might be helpful but it is up to a user to make sure that:

no established TCP connections are present;
file system get snapshotted with post-dump script (and then manually restored before criu restore takes place).

External bind mounts

When dumping LXC or Docker container, one will likely meet those. The --ext-mount-map option should be used for such situations.