Plugins
Linux processes are not things-in-themselves. They tend to cooperate with other parts of the system very actively. When CRIU tries to dump a process having such an external connection, it refuses to make the dump. The reason is simple: without knowing the details of such connections it is impossible to correctly detach the process from its peer on dump and attach back on restore.
Though sometimes we can handle this (this is what --shell-job, --ext-unix-sk and --tcp-established options are about), in many cases we cannot provide a generic solution.
So, in order to address this problem we made CRIU pluggable.
Libraries
A CRIU plugin is a shared library, loaded before dumping or restoring.
Each library can have cr_plugin_init() and cr_plugin_fini() functions for initializing and finalizing. cr_plugin_init() can return a negative value in case of an error.
Headers
The public headers have criu- prefix. The backward compatibility of all functions declared there will be retained.
# ls include/criu-* include/criu-log.h include/criu-plugin.h
Images
CRIU saves images in google protocol buffer (PB) format. We recommend using this format for plugin images as well.
All images are saved in a specified directory and plugins can call criu_get_image_dir() to get the file descriptor on this directory. The file descriptor is used as the directory can be unreachable by path (for example, if processes are restored in a new mount namespace).
Callbacks
When CRIU receives an unsupported object, it enumerates callbacks until one of them doesn't return something other than -ENOSUPP. Usually callbacks return negative code in an error case. The -ENOSUPP code is special — it is returned if the callback is not suitable for the object.
A callback gets an unique identifier for a serialized object. On restore, this identifier is used to request a specific object.
The CRIU tool looks up callbacks by their names. A library may provide any set of callbacks. All callback prototypes are defined in criu-plugins.h.
External unix sockets
A socket is external, if it is dumped without a peer. In this case some state can be present on the other side.
The most popular example of this type of sockets is a D-Bus socket. Only D-Bus daemon knows which events a socket is subscribed to.
int cr_plugin_dump_unix_sk(int sk, int id);
int cr_plugin_restore_unix_sk(int id);The dump callback gets a socket to be dumped. The restore callback returns a restored socket.
External files
External files are files, which are dumped and restored with help plugins.
int cr_plugin_dump_file(int fd, int id);
int cr_plugin_restore_file(int id);The dump callback gets a file descriptor to be dumped. The restore callback returns a restored file.
External bind mounts
These are mount points with target sitting out of the namespace's visible FS tree. For example, this is how LXC people configure the console and ttys for a container. In order to dump and restore those plugin can define calls:
int cr_plugin_dump_ext_mount(char *mountpoint, int id);The mountpoint is the path where the mount is.
The id is an identifier of a mountpoint as seen in /proc/self/mountinfo and that will at restore time be passed to identify the mount to restore (note at that time it will no longer match the proc info!).
int cr_plugin_restore_ext_mount(int id, char *mountpoint, char *old_root, int *is_file)The id is the mountpoint id.
The mountpoint is where the mount should be attached to. Note, that this path may not coincide with the one at dump stage! CRIU may ask to mount it as some different location and will move it to proper place eventually.
The old_root argument is the path to a tree where files visible from the original namespace are. Plugins should get files they want to bind-mount inside the new namespace starting from that path.
The is_file, if not NULL, points to the boolean variable saying whether the mountpoint is a file or a directory. Since in Linux one can bind mount files or directories, CRIU need to know whether the new mount point created by plugin is one of those.
External net devices (links)
When trying to C/R a net namespace with links put outside of it (physical devices assigned or macvlans/vlans), CRIU will call
int cr_plugin_dump_ext_link(int index, int type, char *kind);The index is the link index, type is one of the ARPHRD_* values, and kind is the link driver name.
Restoring external links should happen in setup-namespaces action scripts.
Examples
Here is a small test program that opens /dev/rtc and sets up a timer.
RTC test
Here is a plugin used for dumping and restoring the previous test. criu-rtc.so