Libsoccr

From CRIU
Revision as of 19:58, 19 January 2017 by Avagin (talk | contribs) (→‎Flags)
Jump to navigation Jump to search

Libsoccr is the library that does TCP connection checkpoint and restore. Currently it resides in criu source code (soccr subdirectory) and consists of an API header and a static library. Available since Waxwing release.

Overview

To checkpoint a TCP connection, one should

  1. pause a socket (referenced by descriptor)
  2. get socket data
  3. get contents of two queues
  4. (optionally) resume the socket
  5. close it

To restore a connection, one should

  1. create a socket
  2. pause one
  3. restore unbound data
  4. bind() and/or connect() socket to whatever addresses is appropriate using standard calls
  5. restore noq data
  6. put contents of two queues
  7. restore remaining data

Data types

soccr/soccr.h declares three types:

struct libsoccr_sk
An abstract handler returned by pausing routine that is used as an opaque identifier by the rest of the libsoccr calls
struct libsoccr_sk_data
A structure containing a set of 32-bit values that are returned by getter and that are to be passed as is to the setter call
struct libsoccr_addr
A structure that defines soccaddr for a socket. It's used by set/get calls described further.

Data internals

Inside the libsoccr_sk_data structure there's a special field called flags that gives information about the meaning of some other fields. Though it's not bad just to pass this structure as-is between libsoccr calls, doing some optimization based on flags is OK.

Here are the flags that make sense:

SOCCR_FLAGS_WINDOW == 0x1
When set, indicates that fields snd_wl1, snd_wnd, max_window, rcv_wnd and rcv_wup contain some valuable data (0 is a valid value for any of these)

Calls

Generic

struct libsoccr_sk *libsoccr_pause(int sk)
Pauses the socket and returns a handler for further operations. Any data flow for this socket must be blocked by the caller before this call.
void libsoccr_resume(struct libsoccr_sk *)
Resumes the socket and frees the handler. Data flow unlock may happen before this.
void libsoccr_release(struct libsoccr_sk *)
Only releases the handler and doesn't do anything with the socket itself. Use when saving the state and willing to kill the socket -- call close() on descriptor and this thing on the handler.
void libsoccr_set_log(unsigned int level, void (*fn)(unsigned int level, const char *fmt, ...))
Sets a function that will be called when libsoccr will want to print something on the screen.

Dump

int libsoccr_save(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)
Fills in the data structure with the info get from paused socket. The data_size should be the size of the structure passed. The return code is the size of the structure really filled with data.
char *libsoccr_get_queue_bytes(struct libsoccr_sk *sk, int queue_id, unsigned flags)
Returns a malloc()-ed array with the contents of the queue. The queue can be TCP_RECV_QUEUE or TCP_SEND_QUEUE. The amount of bytes in each is data.inq_len and data.outq_len respectively. The flags may have SOCCR_MEM_EXCL bit set (see below).
union libsoccr_addr *libsoccr_get_addr(struct libsoccr_sk *sk, int self, unsigned flags) (currently unimplemented and returns NULL)
Returns address (self or peer) of the socket. The flags may have SOCCR_MEM_EXCL bit set (see below)

Restore

int libsoccr_restore(struct libsoccr_sk *sk, struct libsoccr_sk_data *data, unsigned data_size)
Restores data on a socket from the data structure. Should be called after the calls below, but before the "resume" one.
int libsoccr_set_queue_bytes(struct libsoccr_sk *sk, int queue, char *buf, unsigned flags)
Puts back the contents of the respective queue. The flags may contain SOCCR_MEM_EXCL.
int libsoccr_set_addr(struct libsoccr_sk *sk, int self, union libsoccr_addr *, unsigned flags)
Sets address (self or peer) of the socket. The flags may contain SOCCR_MEM_EXCL.

Flags

Here are the flags accepted by various calls.

SOCCR_MEM_EXCL
Tells that the memory buffer set or get by the call is to be managed (and should be freed) by the receiving side. In case of a get call the receiving side is the caller program, meaning it has to free() the buffer. In case of a set call the receiver is libsoccr, meaning libsoccr has to free() it.

Example

/* Dump a tcp socket */

dst_let = sizeof(addr);
if (getsockname(sock, (struct sockaddr *) &addr, &dst_let)) {
        pr_perror("connect");
        return -1;
}
dst_let = sizeof(addr);
if (getpeername(sock, (struct sockaddr *) &dst, &dst_let)) {
        pr_perror("connect");
        return -1;
}

so = libsoccr_pause(sock);

dsize = libsoccr_save(so, &data, sizeof(data));
if (dsize < 0) {
        pr_perror("libsoccr_save");
        return -1;
}
close(sock);


/* Restore a tcp socket */
rst = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (rst == -1)
        return -1;


so_rst = libsoccr_pause(rst);
libsoccr_set_addr(so_rst, 1, &addr, 0);
libsoccr_set_addr(so_rst, 0, &dst, 0);

queue = libsoccr_get_queue_bytes(so, TCP_RECV_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_RECV_QUEUE, queue, SOCCR_MEM_EXCL);
queue = libsoccr_get_queue_bytes(so, TCP_SEND_QUEUE, SOCCR_MEM_EXCL);
libsoccr_set_queue_bytes(so_rst, TCP_SEND_QUEUE, queue, SOCCR_MEM_EXCL);

ret = libsoccr_restore(so_rst, &data, dsize);
if (ret)
        return -1;

libsoccr_resume(so_rst);
libsoccr_resume(so);