RPC

CRIU-RPC is a remote procedure call (RPC) protocol which uses Google Protocol Buffers to encode its calls. The requests are served by CRIU service launched with criu service command. It uses a SEQPACKET Unix domain socket listening at /var/run/criu-service.socket as a transport.

Protobuf messages

criu_req/criu_resp -- wrappers for requests/responses. They are to be used for transferring messages and needed to provide compatibility with an older versions of rpc. Field type in them _must_ be set accordingly to type of request/response that is stored. Types of request/response are defined in enum criu_req_type.

Request

criu_req

message criu_req {
	required criu_req_type type	= 1;

	optional criu_opts opts		= 2;
}

criu_req_type

There are 8 request/response types for now. Comments show cmdline analogs.

enum criu_req_type {
	EMPTY		= 0;
	DUMP		= 1; /* criu dump */
	RESTORE		= 2; /* criu restore */
	CHECK		= 3; /* criu check */
	PRE_DUMP	= 4; /* criu pre-dump */
	PAGE_SERVER	= 5; /* criu page-server */

	NOTIFY		= 6;

	CPUINFO_DUMP	= 7; /* criu cpuinfo dump */
	CPUINFO_CHECK	= 8; /* criu cpuinfo check */
}

criu_opts

It is used to store options.

message criu_opts {
	required int32			images_dir_fd	= 1;
	optional int32			pid		= 2; /* if not set on dump, will dump requesting process */

	optional bool			leave_running	= 3;
	optional bool			ext_unix_sk	= 4;
	optional bool			tcp_established	= 5;
	optional bool			evasive_devices	= 6;
	optional bool			shell_job	= 7;
	optional bool			file_locks	= 8;
	optional int32			log_level	= 9 [default = 2];
	optional string			log_file	= 10; /* No subdirs are allowed. Consider using work-dir */

	optional criu_page_server_info	ps		= 11;

	optional bool			notify_scripts	= 12;

	optional string			root		= 13;
	optional string			parent_img	= 14;
	optional bool			track_mem	= 15;
	optional bool			auto_dedup	= 16;

	optional int32			work_dir_fd	= 17;
	optional bool			link_remap	= 18;
	repeated criu_veth_pair		veths		= 19;

	optional uint32			cpu_cap		= 20 [default = 0xffffffff];
	optional bool			force_irmap	= 21;
	repeated string			exec_cmd	= 22;

	repeated ext_mount_map		ext_mnt		= 23;
	optional bool			manage_cgroups	= 24;
	repeated cgroup_root		cg_root		= 25;

	optional bool			rst_sibling	= 26; /* swrk only */
}

If no pid is set and type is DUMP, CRIU will dump client process by default. Note: Whole tree <pid> must have the same uid, as a client, or client's uid must be == 0, otherwise CRIU won't dump nothing at all.

Only images_dir_fd is required, all other fields may not be set. Client must open directory for/with images by himself and set images_dir_fd to it's fd. CRIU will open /proc/<client's_pid>/fd/<images_dir_fd>, so it will work, if client is in another namespace.

The logic of setting request is the same as when setting options in console. Here is an example:

#criu restore -D /path/to/imgs_dir -v4 -o restore.log

is equal to:

request.type = RESTORE;

request.opts.imgs_dir_fd	= open("/path/to/imgs_dir")
request.opts.log_level		= 4
request.opts.log_file		= "restore.log"

criu_page_server_info

Stores info about page-server.

message criu_page_server_info {
	optional string		address	= 1; /* bind address -- if not set 0.0.0.0 is used */
	optional int32		port	= 2; /* bind port -- if not set on request, autobind is used and port is returned in response */
	optional int32		pid	= 3; /* page-server pid -- returned in response */
	optional int32		fd	= 4; /* could be used to inherit fd by page-server */
}

criu_veth_pair

TODO

message criu_veth_pair {
	required string		if_in	= 1;
	required string		if_out	= 2;
};

ext_mount_map

TODO

message ext_mount_map {
	required string		key	= 1;
	required string		val	= 2;
};

cgroup_root

TODO

message cgroup_root {
	optional string		ctrl	= 1;
	required string		path	= 2;
};

Response

criu_resp

message criu_resp {
	required criu_req_type		type		= 1;
	required bool			success		= 2;

	optional criu_dump_resp		dump		= 3;
	optional criu_restore_resp	restore		= 4;
	optional criu_notify		notify		= 5;
	optional criu_page_server_info	ps		= 6;
}

Field "success" reports result of processing request, while criu_***_resp store some request-specific information. The response type is set to the corresponding request type or to EMPTY to report a "generic" error.

criu_dump_resp

criu_dump_resp is used to store dump response from CRIU.

message criu_dump_resp {
	optional bool restored		= 1;
}

Field "restored" is set to "true" if process was restored.

criu_restore_resp

message criu_restore_resp {
	required int32 pid		= 1;
}

Field "pid" is set to the PID of the restored process.

criu_notify

TODO

message criu_notify {
	optional string script		= 1;
	optional int32	pid		= 2;
}

Run

Server

On a server side, CRIU creates SOCK_SEQPACKET Unix socket and listens for connections on it. After receiving criu_req, CRIU processes it, do what is requested and sends criu_resp with set request-specific criu_***_resp field back. If CRIU gets unknown type of request, it will return criu_resp with type == EMPTY and success == false.

To launch service server, run:

#criu service [options]

Options accepted by service are

--address <path>

is where to put listening socket

--pid-file <path>

is where to write pid of service process

--daemon

tells service to daemonize

-o <file>

says where to write logs

-v[N]

sets the log-level

systemd

If you are running systemd you can make service start and operate automatically. First do

make install

to make files criu.service and criu.socket appear in systemd configs (/lib/systemd/system/). Then

systemctl start criu.socket

to get /var/run/criu-service.socket, and

systemctl enable criu.socket

to make /var/run/criu-service.socket available at boot.

Client

Client, in its turn, must connect to service socket, send criu_req with request in it, and wait for a criu_resp with response. You can find examples of client programs in C and Python in test/rpc/.

With RPC facilities one can perform a self dump.

There's a library that implements simple wrappers on top of RPC.

Security

See Security.