Introduction
Linux processes are not things-in-themselves. They tend to cooperate with other parts of the system very actively. When CRIU tries to dump a process, that has such an external connection -- it refuses to make the dump. The reason is simple -- without knowing the details of such connections it's impossible to correctly detach the process from its peer on dump and attach back on restore.
Though sometimes we can handle this (this is what --shell-job, --ext-unix-sk and --tcp-established options are about), in many cases we cannot provide generic solution.
So, in order to address this problem we make CRIU pluggable. //by xemul@
Libraries
A CRIU plugin is shared library, which is loaded before dumping or restoring.
Each library can have cr_plugin_init() and cr_plugin_fini() functions for initializing and finalizing. cr_plugin_init() can return a negative value in an error case.
Headers
The public headers have prefix "criu-". The backward compatibility of all functions, which are declared there, will be saved.
# ls include/criu-* include/criu-log.h include/criu-plugin.h
Images
CRIU saves images in google protocol buffer format (PB format). We recomend to use this format for plugin images too.
All images are saved in a specified directory and plugins can call criu_get_image_dir() to get the file descriptor on this directory. The file descriptor is used, because the directory can be unreachable by path. For example if processes are restored in a new mount namespace.
Callbacks
When CRIU receives an unsupported object, it enumerates callbacks until one of them doesn't return something other than -ENOSUPP. Usually callbacks returns negative code in an error case. The -ENOSUPP code is a special one. It is returned, if a callback is not suitable for the object.
A callback gets an unique identificator for a serialized object. On restore this identificator is used to request a specific object.
The CRIU tool looks up callback by their names. A library may provide any set of callbacks. All callback prototypes are defined in criu-plugins.h.
External unix sockets
A socket is external, if it's dumped without a peer. In this case some state can be hidden on another side.
The most popular example of this type of sockets is a D-Bus socket. Only D-Bus daemon knows on which events a socket is subscribed.
int cr_plugin_dump_unix_sk(int sk, int id)
int cr_plugin_restore_unix_sk(int id)
The dump callback gets a socket, which must be dumped. The restore callback returns a restored socket.
External files
External files are files, which are dumped and restored with help plugins.
int cr_plugin_dump_file(int fd, int id)
int cr_plugin_restore_file(int id)
The dump callback gets a file descriptor, which must be dumped. The restore callback returns a restored file.
External bind mounts
These are mount points with target sitting out of the namespace's visible FS tree. E.g. this is how LXC people configure the console and ttys for a container. In order to dump and restore those plugin can define calls:
int cr_plugin_dump_ext_mount(char *mountpoint, int id)
The mountpoint is the path where the mount is.
The id is an identifier of a mountpoint as seen in /proc/self/mountinfo and that will at restore time be passed to identify the mount to restore (at that time it will no longer match the proc info!).
int cr_plugin_restore_ext_mount(int id, char *mountpoint, char *old_root, int *is_file)
The id is the mountpoint id.
The mountpoint is where the mount should be attached to. Note, that this path may not coincide with the one at dump stage! CRIU may ask to mount it as some different location and will move it to proper place eventually.
The old_root argument is the path to a tree where files visible from the original namespace are. Plugins should get files they want to bind-mount inside the new namespace starting from that path.
The is_file if not NULL points to the boolean variable saying whether the mountpoint is file or directory. Since in Linux one can bind mount files or directories, CRIU need to know whether the new mount point, created by plugin, is one of those.
External net devices (links)
When trying to C/R a net namespace with links put outside of it (physical devices assigned or macvlans/vlans) CRIU will call the
int cr_plugin_dump_ext_link(int index, int type, char *kind);
callback. The index is the link index, type is one of the ARPHRD_ value and kind is the link driver name.
Restoring external links should happen in "setup-namespaces" action scripts.
Examples
Here is a small test program, which opens /dev/rtc and sets up a timer. RTC test
Here is a plugin, which is used for dumping and restoring the previous test. criu-rtc.so