2,257
edits
Changes
Jump to navigation
Jump to search
Line 1:
Line 1: − + + + + + + + + + + + + + + + + − Most of the crtools image files begin with 32-bit magic cookie followed by zero or more entries of the same type (not size!). Most of the entries are encoded in the Google Protocol Buffers format and each is preceded with 32-bit entry size value (not including this 32-bit value itself). Optionally each entry may be followed by extra payload which depends on the entry type.+ + − + + + + − + Line 14:
Line 33: + + + + + + + + + + + + + + + + + + + + − + − Images encoded with ProtocolBuffers can be one of+ + − + Line 36:
Line 76: + + + + + + + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + − + + + + + + + + + + + + + + + + + + + + + + + + + − + − + − + − There are, however, images which are not in ProtocolBuffers.+ + − ; Memory dumps+ − + − <pre>+ − IMAGE ::= MAGIC [ ADDRESS DATA ]+ − ADDRESS ::= "64-bit virtual address"+ − DATA ::= "4K bytes blob (i.e. -- memory page)"+ − </pre>+ + + +
no edit summary
The criu utility dumps the state of processes/containers into a set of image files. This article describes the format of them.
The criu utility dumps the state of processes/containers into a set of image files. This article describes the format of them.
== General format description ==
== Types of image files ==
CRIU images can be in one of the following formats
* criu specific images in google protocol buffer format (PB format)
* criu specific images with binary data in it
* image files in 3rd party format (a.k.a. raw images)
== Images in criu-specific format ==
All criu-specific image files begin with 2 32-bit magic cookies. The first cookie is the type of file (see below) the second is the optional sub-type of image. Images in PB format are followed by zero or more entries of the same type (not size!), each entry is preceded with 32-bit entry size value (not including this 32-bit value itself). Optionally each entry may be followed by extra payload which depends on the entry type.
Currently there are 3 types of images
; Inventory file
: This is the image file describing the set. It doesn't have sub-type magic.
; Image file
: Regular image. Most of the text below is about these files.
IOW image files look like
; Auxiliary file
: File that is not image, but criu generates one and it happens to be in protobuf format too. For now we have only stats and irmap cache files of that type. They also have sub-type magic.
IOW protocol-buffers image files look like
<pre>
<pre>
IMAGE_FILE ::= MAGIC { ENTRY }
IMAGE_FILE ::= MAGIC [MAGIC_2] { ENTRY }
ENTRY ::= SIZE PAYLOAD [ EXTRA ]
ENTRY ::= SIZE PAYLOAD [ EXTRA ]
PAYLOAD ::= "message encoded in ProtocolBuffer format"
PAYLOAD ::= "message encoded in ProtocolBuffer format"
MAGIC ::= "32 bit integer"
MAGIC ::= "32 bit integer"
MAGIC_2 ::= "32 bit integer"
SIZE ::= "32 bit integer, equals the PAYLOAD length"
SIZE ::= "32 bit integer, equals the PAYLOAD length"
</pre>
</pre>
Or, you can visualize it like
{| class="wikitable"
|-
! Type !! Size, bytes
|-
| Magic || 4
|-
| Size0 || 4
|-
| Message0 || Size0
|-
| ... || ...
|-
| SizeN || 4
|-
| MessageN || SizeN
|}
The amount of entries in a image file depends on the type of file.
The amount of entries in a image file depends on the type of file.
== Types of image files ==
=== Images with PB data ===
Such images can be one of
; Array image files
; Array image files
: In these files the amount of entries can be any. You should read the image file up to the EOF to find out the exact number.
: In these files the amount of entries can be any. You should read the image file up to the EOF to find out the exact number.
; Single-entry image files
; Single-entry image files
: In these files exactly one entry is stored.
: In these files exactly one entry is stored.
A file type can be guessed by the magic. The description of the entries in ProtocolBuffers language are in <type>.proto files.
A file type can be guessed by the magic. The description of the entries in ProtocolBuffers language are in respective .proto files which reside in <code>protobuf/</code> directory in the source tree.
{|class="wikitable sortable"
{|class="wikitable sortable"
! description
! description
! extra payload
! extra payload
! describing proto file
|-
| inventory || single-entry || Top level description of images || - || inventory.proto
|-
| fdinfo || array || [[Fdinfo-engine|Open file descriptors]] || - || fdinfo.proto
|-
| reg-files || array || Paths to [[:Category:Files|files]] opened with <code>open(2)</code> syscall || - || regfile.proto
|-
|-
| inventory || single-entry || Top level description of images || -
| eventfd || array || Eventfd file information || - || eventfd.proto
|-
|-
| fdinfo || array || Open file descriptors || -
| eventpoll || array || Eventpoll file information || - || eventpoll.proto
|-
|-
| regfile || array || Paths to files opened with <code>open(2)</code> syscall || -
| eventpoll-tfd || array || Target file descriptors of eventpoll fds (merged into above) || - || eventpoll.proto
|-
|-
| eventfd || array || Eventfd file information || -
| inotify || array || Inotify file information || - || intotify.proto
|-
|-
| eventpoll || array || Eventpoll file information || -
| inotify-wd || array || Watch descriptors of inotify fds (merged into above) || - || inotify.proto
|-
|-
| eventpoll-tfd || array || Target file descriptors of eventpoll fds || -
| signalfd || array || signalfd info || - || signalfd.proto
|-
|-
| inotify-file || array || Inotify file information || -
| core || single-entry || Core process info and (name, sigmask, itimers, etc.) arch-dependent information (registers, etc.) || - || core.proto
|-
|-
| inotify-wd || array || Watch descriptors of inotify fds || -
| mm || single-entry || [[Memory dumping and restoring|Address space]] information (VMAs, segments, exe file, etc.) || - || mm.proto
|-
|-
| signalfd || array || signalfd info
| pipes || array || Pipes information || - || pipe.proto
|-
|-
| core || single-entry || Arch-dependent information (registers, etc.) || -
| pipes-data || array || Contents of pipes || <code>entry.bytes</code> bytes of data sitting in a pipe || pipe-data.proto
|-
|-
| mm || single-entry || Address space generic information (segments, exe file, etc.) || -
| fifo || array || FIFO information || - || fifo.proto
|-
|-
| vmas || array || Virtual mappings (<code>mmap(2)</code>) || -
| fifo-data || array || Contents of FIFOs || same as in pipes-data || pipe-data.proto
|-
|-
| pipes || array || Pipes information || -
| pstree || array || Process [[tree after restore|tree linkage]] || - || pstree.proto
|-
|-
| pipes-data || array || Contents of pipes || <code>entry.bytes</code> bytes of data sitting in a pipe
| ids || single || IDs of objects (mm, files, sihand, etc.) and namespaces || - || core.proto
|-
|-
| fifo || array || FIFO information || -
| sigacts || array || Signal handling map || - || sa.proto
|-
|-
| fifo-data || array || Contents of FIFOs || same as in pipes-data
| unixsk || array || [[Unix sockets]] || - || sk-unix.proto
|-
|-
| pstree || array || Process tree linkage and IDs || -
| inetsk || array || PF_INET sockets, both IPv4 and IPv6 || - || sk-inet.proto
|-
|-
| sigacts || array || Signal handling map || -
| sk-queues || array || Contents of socket queues || <code>entry.length</code> bytes of data, one entry per packet || sk-packet.proto
|-
|-
| unixsk || array || Unix domain sockets || -
| itimers || array || Interval timers state (merged into core image) || - || itimer.proto
|-
|-
| inetsk || array || PF_INET sockets, both IPv4 and IPv6 || -
| creds || single-entry || Task credentials: uids, gids, caps, etc. || - || creds.proto
|-
|-
| sk-queues || array || Contents of socket queues || <code>entry.length</code> bytes of data, one entry per packet
| fs || single-entry || Chroot and chdir information || - || fs.proto
|-
|-
| itimers || array || Interval timers state || -
| remap-fpath || array || File paths remaps (e.g. for [[invisible files]]) || - || remap-file-path.proto
|-
|-
| creds || single-entry || Task credentials: uids, gids, caps, etc. || -
| ghost-file || single-entry || Ghost [[invisible files]] || Right after the entry up to the EOF goes the contents of the file || ghost-file.proto
|-
|-
| fs || single-entry || Chroot and chdir information || -
| tcp-stream || single-entry || [[TCP connection]] state (including data in queues) || <code>entry.inq_len</code> bytes of in-queue data followed by <code>entry.outq_len</code> bytes of out-queue data || tcp-stream.proto
|-
|-
| path-remap || array || File paths remaps || -
| mountpoints || array || [[Mountpoints]] information || - || mnt.proto
|-
|-
| ghost-file || single-entry || Ghost files (those, not visible from FS, but still used by tasks) || Right after the entry up to the EOF goes the contents of the file
| utsns || single-entry || Uname nodename and domainname of a UTS namespace || - || utsns.proto
|-
|-
| tcp-stream || single-entry || TCP connection state (including data in queues) || -
| tty || array || Information about opened [[TTYs]] || - || tty.proto
|-
|-
| mountpoints || array || Mountpoints information || -
| tty-info || array || Termios and similar stuff about [[TTYs]] || - || tty.proto
|-
|-
| utsns || single-entry || Uname nodename and domainname of a UTS namespace || -
| packetsk || array || Info about PF_PACKET sockets || - || packet-sock.proto
|-
|-
| tty || array || Information about opened tty-s
| netdev || array || Info about [[:Category:Network|network]] devices || - || netdev.proto
|}
=== Images with memory dumps ===
''Main article: [[memory dumps]]''.
Anonymous memory contents (both private and shared) is stored in two types of images:
; Pagemap files
: These files contain info about which virtual regions are populated with data. The file is a set of protobuf messages.
{{Note| Even though pagemap is an array kind of image (and can be included to the previous type), first pb message is of type pagemap_head and all the following ones are of type pagemap_entry.}}
; Pages files
: These contain 4k pages that are to be put into the memory according to the pagemap.
== Raw images ==
These images contain data that were collected by criu with the help of some external tools.
{|class="wikitable sortable"
|-
! Name
! Tool supporting the format
! Description
|-
|-
| tty-info || array || Termios and similar stuff about tty-s
| ifaddr || ip from iproute2 || IP addresses on network devices
|-
|-
| packetsk || array || Info about PF_PACKET sockets
| route || ip from iproute2 || Routing tables
|-
|-
| netdev || array || Info about network devices
| tmpfs || tar + gzip || Contents of a tmpfs filesystem
|}
|}
== Notes about protobuf ==
We have a registered field number (1018) for [https://developers.google.com/protocol-buffers/docs/proto#options custom options] of all kinds. See protobuf/opts.proto for more info.
== See also ==
: These files contain memory dumps. The format is
* [[CRIT]]: a tool to decode images to a human readable format
* [[What's bad with V1 images]]
* [[Image field merging]]
* [[Memory dumps]]
[[Category:Development]]
[[Category:Images]]
[[Category:Outdated]]