Difference between revisions of "CRIT"

From CRIU
Jump to navigation Jump to search
(11 intermediate revisions by 3 users not shown)
Line 4: Line 4:
 
== Usage ==
 
== Usage ==
 
<pre>
 
<pre>
−
usage: crit [-h] [-i IN] [-o OUT] [-f {raw,nice}] {decode,encode}
+
usage: crit [-h] {decode,encode,info,x,show} ...
  
 
CRiu Image Tool
 
CRiu Image Tool
  
 
positional arguments:
 
positional arguments:
−
   {decode,encode}       decode/encode - convert criu image from/to binary type
+
   {decode,encode,info,x,show}
−
                        to/from json
+
                        Use crit CMD --help for command-specific help
 +
    decode             convert criu image from binary type to json
 +
    encode             convert criu image from json type to binary
 +
    info                show info about image
 +
    x                  explore image dir
 +
    show                convert criu image from binary to human-readable json
  
 
optional arguments:
 
optional arguments:
 
   -h, --help            show this help message and exit
 
   -h, --help            show this help message and exit
−
  -i IN, --in IN        input file (stdin by default)
 
−
  -o OUT, --out OUT    output file (stdout by default)
 
−
  --pretty              multi-line with indentation and some fields printed in more human-readable format
 
 
</pre>
 
</pre>
  
 
=== Pretty output ===
 
=== Pretty output ===
  
−
By default CRIT prints JSON text in one line. This can be read by any further JSON-aware tool for parsing. For human eyes it's more convenient to read JSON multi-line with indentation. However, CRIT does a little bit more and prints some fields in even more pretty manner.
+
Command <decode> prints JSON text in one line. This can be read by any further JSON-aware tool for parsing. For human eyes it's more convenient to read JSON multi-line with indentation. CRIT can do this when using the <code>show</code> action or the <code>--pretty</code> option.
 +
 
 +
Wend doing pretty optput CRIT does a little bit more, than just multiline indented JSON.
  
 
; Addresses and registers
 
; Addresses and registers
Line 31: Line 35:
 
; IP addresses
 
; IP addresses
 
: By default those a printed in decimal, but the "1.2.3.4" for v4 or "::1" for v6 can be seen in the --pretty mode.
 
: By default those a printed in decimal, but the "1.2.3.4" for v4 or "::1" for v6 can be seen in the --pretty mode.
 +
 +
; Symbolic names for flags
 +
: Some known bit sets (e.g. MAP_PRIVATE, MAP_ANONYMOUS, etc. for vma->flags) are shown with names.
 +
 +
; Device numbers
 +
: If the field is known to be of <code>dev_t</code> type, it's printed in the <code>major:minor</code> manner
  
 
== Functionality ==
 
== Functionality ==
Line 36: Line 46:
 
=== Convert images to JSON and back ===
 
=== Convert images to JSON and back ===
  
−
{{Status|ready}}
 
  
−
This is the replacement for (rather nasty) criu show code. Also this is the way to edit the images before restoring from them.
+
This is the replacement for (rather nasty) <code>criu show</code> code. Also this is the way to edit the images before restoring from them.
  
 
It uses [https://developers.google.com/protocol-buffers/docs/reference/python/google.protobuf.text_format-module text_format] as a human-readable format for protobuf messages.
 
It uses [https://developers.google.com/protocol-buffers/docs/reference/python/google.protobuf.text_format-module text_format] as a human-readable format for protobuf messages.
Line 44: Line 53:
 
The output file is structured in the following way.
 
The output file is structured in the following way.
  
−
With "--format raw":
+
Without "--pretty":
 
<pre>
 
<pre>
 
{ "magic" : "FOO", "entries" : [{"foo": "bar", "bar": "foo", "extra": "abc"}, {"foo": "bar", "bar" : "foo", "extra" : "abc"} ]}
 
{ "magic" : "FOO", "entries" : [{"foo": "bar", "bar": "foo", "extra": "abc"}, {"foo": "bar", "bar" : "foo", "extra" : "abc"} ]}
 
</pre>
 
</pre>
  
−
With "--format nice"(default for stdout):
+
With "--pretty":
 
<pre>
 
<pre>
 
{
 
{
Line 69: Line 78:
  
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
−
<b>Example("crit decode -i core-20700.img")</b>
+
<b>Example("crit decode -i core-5679.img --pretty")</b>
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
 
<pre>
 
<pre>
Line 201: Line 210:
 
                     },  
 
                     },  
 
                     &quot;xmm_space&quot;: [
 
                     &quot;xmm_space&quot;: [
−
                        1953064751,
 
−
                        1769104175,
 
−
                        1702113141,
 
−
                        1664054387,
 
 
                         0,  
 
                         0,  
 +
                        4278190080,
 +
                        4294967295,
 +
                        4294967295,
 +
                        1701145715,
 +
                        3219568,
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
−
                         4294901760,
+
                         33,  
−
                        255,  
 
 
                         0,  
 
                         0,  
 
                         792358505,  
 
                         792358505,  
Line 229: Line 238:
 
                         65,  
 
                         65,  
 
                         0,  
 
                         0,  
−
                         21771440,  
+
                         37049520,
 +
                        0,
 +
                        37049632,
 +
                        0,
 +
                        0,
 +
                        0,
 +
                        0,  
 
                         0,  
 
                         0,  
−
                        21771552,
 
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
Line 237: Line 251:
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
−
                        4278190080,
 
−
                        4294967295,
 
−
                        4294967040,
 
−
                        4294967295,
 
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
Line 246: Line 256:
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
−
                        4278190080,
 
−
                        4294967040,
 
−
                        4294967295,
 
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
 +
                        0,
 +
                        4294901760,
 
                         0,  
 
                         0,  
 
                         0,  
 
                         0,  
Line 271: Line 280:
 
                 &quot;clear_tid_addr&quot;: 0,  
 
                 &quot;clear_tid_addr&quot;: 0,  
 
                 &quot;gpregs&quot;: {
 
                 &quot;gpregs&quot;: {
−
                     &quot;gs&quot;: 0,  
+
                     &quot;gs&quot;: &quot;0x0&quot;,  
−
                     &quot;ip&quot;: 140137968588056,  
+
                     &quot;ip&quot;: &quot;0x7f172cf1ea04&quot;,  
−
                     &quot;cx&quot;: 18446744073709551615,  
+
                     &quot;cx&quot;: &quot;0xffffffffffffffff&quot;,  
−
                     &quot;cs&quot;: 51,  
+
                     &quot;cs&quot;: &quot;0x33&quot;,  
−
                     &quot;ax&quot;: 61,  
+
                     &quot;ax&quot;: &quot;0x38&quot;,  
−
                     &quot;orig_ax&quot;: 61,  
+
                     &quot;orig_ax&quot;: &quot;0x38&quot;,  
−
                     &quot;di&quot;: 18446744073709551615,  
+
                     &quot;di&quot;: &quot;0x1200011&quot;,  
−
                     &quot;es&quot;: 0,  
+
                     &quot;es&quot;: &quot;0x0&quot;,  
−
                     &quot;gs_base&quot;: 0,  
+
                     &quot;gs_base&quot;: &quot;0x0&quot;,  
−
                     &quot;r14&quot;: 0,  
+
                     &quot;r14&quot;: &quot;0x0&quot;,  
−
                     &quot;r15&quot;: 4294967295,  
+
                     &quot;r15&quot;: &quot;0x2355e00&quot;,  
−
                     &quot;r12&quot;: 21789168,  
+
                     &quot;r12&quot;: &quot;0x7ffffdbf74f0&quot;,  
−
                     &quot;r13&quot;: 0,  
+
                     &quot;r13&quot;: &quot;0x0&quot;,  
−
                     &quot;r10&quot;: 0,  
+
                     &quot;r10&quot;: &quot;0x7f172d83d9d0&quot;,  
−
                     &quot;r11&quot;: 582,  
+
                     &quot;r11&quot;: &quot;0x246&quot;,  
−
                     &quot;fs_base&quot;: 140137978173184,  
+
                     &quot;fs_base&quot;: &quot;0x7f172d83d700&quot;,  
−
                     &quot;bp&quot;: 2560,  
+
                     &quot;bp&quot;: &quot;0x7ffffdbf7530&quot;,  
−
                     &quot;dx&quot;: 0,  
+
                     &quot;dx&quot;: &quot;0x0&quot;,  
−
                     &quot;bx&quot;: 0,  
+
                     &quot;bx&quot;: &quot;0x0&quot;,  
−
                     &quot;ds&quot;: 0,  
+
                     &quot;ds&quot;: &quot;0x0&quot;,  
−
                     &quot;ss&quot;: 43,  
+
                     &quot;ss&quot;: &quot;0x2b&quot;,  
−
                     &quot;sp&quot;: 140736135950616,  
+
                     &quot;sp&quot;: &quot;0x7ffffdbf74f0&quot;,  
−
                     &quot;r8&quot;: 21789168,  
+
                     &quot;r8&quot;: &quot;0x0&quot;,  
−
                     &quot;r9&quot;: 0,  
+
                     &quot;r9&quot;: &quot;0x0&quot;,  
−
                     &quot;fs&quot;: 0,  
+
                     &quot;fs&quot;: &quot;0x0&quot;,  
−
                     &quot;si&quot;: 140736135950656,  
+
                     &quot;si&quot;: &quot;0x0&quot;,  
−
                     &quot;flags&quot;: 582
+
                     &quot;flags&quot;: &quot;0x246&quot;
 
                 }
 
                 }
 
             },  
 
             },  
Line 323: Line 332:
 
                 &quot;cg_set&quot;: 1,  
 
                 &quot;cg_set&quot;: 1,  
 
                 &quot;signals_s&quot;: {},  
 
                 &quot;signals_s&quot;: {},  
−
                 &quot;blk_sigset&quot;: 65536,  
+
                 &quot;blk_sigset&quot;: &quot;0x10002&quot;,  
 
                 &quot;exit_code&quot;: 0,  
 
                 &quot;exit_code&quot;: 0,  
 
                 &quot;rlimits&quot;: {
 
                 &quot;rlimits&quot;: {
Line 352: Line 361:
 
                         },  
 
                         },  
 
                         {
 
                         {
−
                             &quot;max&quot;: 62845,  
+
                             &quot;max&quot;: 62844,  
−
                             &quot;cur&quot;: 62845
+
                             &quot;cur&quot;: 62844
 
                         },  
 
                         },  
 
                         {
 
                         {
Line 372: Line 381:
 
                         },  
 
                         },  
 
                         {
 
                         {
−
                             &quot;max&quot;: 62845,  
+
                             &quot;max&quot;: 62844,  
−
                             &quot;cur&quot;: 62845
+
                             &quot;cur&quot;: 62844
 
                         },  
 
                         },  
 
                         {
 
                         {
Line 407: Line 416:
 
=== Generate core files out of task images ===
 
=== Generate core files out of task images ===
  
−
{{Status|in progress}}
+
Moved into separate project called [[criu-coredump]].
  
−
All the needed information is in core-$pid.img and pagemap-$pid.img
+
=== Show images statistics ===
  
−
=== On-the-fly conversion ===
+
E.g. -- total number of processes, files, memory, sockets, etc.
 +
Use
 +
<pre>
 +
crit x $directory_with_images <explore type>
 +
</pre>
  
−
{{Status|not ready}}
+
Supported explorers are
 +
;'ps'
 +
:to show process tree
  
−
There's an idea to make CRIU spawn CRIT and read images "through" it, to allow for at-the-restore-time modifications
+
;'fds'
 +
:to show files used by tasks
  
−
=== Convert between different image versions ===
+
;'mems'
 +
:to show memory mappings info
  
−
{{Status|not ready}}
+
== Planned functionality ==
  
−
Right now we store the images version in inventory.img and collect info about [[what's bad with V1 images]]. If some day we have v2, CRIT will convert from v1. And for backward compatibility we'll use on-the-fly conversion when restoring from old images.
+
=== On-the-fly conversion ===
  
−
=== Show images statistics ===
+
There's an idea to make CRIU spawn CRIT and read images "through" it, to allow for at-the-restore-time modifications. For details, see https://github.com/xemul/criu/issues/59.
  
−
{{Status|not ready}}
+
=== Convert between different image versions ===
  
−
E.g. -- total number of processes, files, memory, sockets, etc.
+
Right now we store the images version in inventory.img and collect info about [[what's bad with V1 images]]. If some day we have v2, CRIT will convert from v1. And for backward compatibility we'll use on-the-fly conversion when restoring from old images.
  
 
=== Check/validate images ===
 
=== Check/validate images ===
−
 
−
{{Status|not ready}}
 
  
 
Check that
 
Check that
 
* all images are present
 
* all images are present
 
* the inter-images IDs are in consistent state
 
* the inter-images IDs are in consistent state
 +
 +
For details, see https://github.com/xemul/criu/issues/44.
 +
 +
=== [[Anonymize image files]] ===
 +
 +
Since images contain raw memory dumps people may refuse to send us images for debugging. Need to anonymize them, i.e. -- remove this sensitive information.
  
 
[[Category:Images]]
 
[[Category:Images]]
 
[[Category:API]]
 
[[Category:API]]

Revision as of 10:17, 27 March 2019

This page describes the CRiu Image Tool. CRIT is a feature-rich replacement for existing "criu show". It is written completely in Python, so it is quite easy to read the code and extend its features.

Usage

usage: crit [-h] {decode,encode,info,x,show} ...

CRiu Image Tool

positional arguments:
  {decode,encode,info,x,show}
                        Use crit CMD --help for command-specific help
    decode              convert criu image from binary type to json
    encode              convert criu image from json type to binary
    info                show info about image
    x                   explore image dir
    show                convert criu image from binary to human-readable json

optional arguments:
  -h, --help            show this help message and exit

Pretty output

Command <decode> prints JSON text in one line. This can be read by any further JSON-aware tool for parsing. For human eyes it's more convenient to read JSON multi-line with indentation. CRIT can do this when using the show action or the --pretty option.

Wend doing pretty optput CRIT does a little bit more, than just multiline indented JSON.

Addresses and registers
VM addresses and core.img register values are all printed in hex. Since JSON doesn't support this form of numbers, such fields are encoded as strings.
Bit-fields
Such things as flags and masks (e.g. sig-block mask) are also better understood when written in hex, so CRIT does this.
IP addresses
By default those a printed in decimal, but the "1.2.3.4" for v4 or "::1" for v6 can be seen in the --pretty mode.
Symbolic names for flags
Some known bit sets (e.g. MAP_PRIVATE, MAP_ANONYMOUS, etc. for vma->flags) are shown with names.
Device numbers
If the field is known to be of dev_t type, it's printed in the major:minor manner

Functionality

Convert images to JSON and back

This is the replacement for (rather nasty) criu show code. Also this is the way to edit the images before restoring from them.

It uses text_format as a human-readable format for protobuf messages.

The output file is structured in the following way.

Without "--pretty": 

{ "magic" : "FOO", "entries" : [{"foo": "bar", "bar": "foo", "extra": "abc"}, {"foo": "bar", "bar" : "foo", "extra" : "abc"} ]}

With "--pretty": 

{
    "magic" : "FOO",
    "entries" : [
        {
            "foo" : "bar",
            "bar" : "foo",
            "extra" : "abc"
        },
        {
            "foo" : "bar",
            "bar" : "foo",
            "extra" : "abc"
        }
    ]
}

Example("crit decode -i core-5679.img --pretty") 

{
    "magic": "CORE", 
    "entries": [
        {
            "mtype": "X86_64", 
            "thread_core": {
                "futex_rla_len": 24, 
                "sched_policy": 0, 
                "sched_nice": 0, 
                "futex_rla": 0, 
                "signals_p": {}, 
                "sas": {
                    "ss_size": 0, 
                    "ss_sp": 0, 
                    "ss_flags": 2
                }
            }, 
            "thread_info": {
                "fpregs": {
                    "st_space": [
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0
                    ], 
                    "fop": 0, 
                    "rdp": 0, 
                    "twd": 0, 
                    "mxcsr": 8064, 
                    "swd": 0, 
                    "rip": 0, 
                    "xsave": {
                        "ymmh_space": [
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0, 
                            0
                        ], 
                        "xstate_bv": 2
                    }, 
                    "xmm_space": [
                        0, 
                        4278190080, 
                        4294967295, 
                        4294967295, 
                        1701145715, 
                        3219568, 
                        0, 
                        0, 
                        0, 
                        0, 
                        33, 
                        0, 
                        792358505, 
                        1953460082, 
                        1852400175, 
                        0, 
                        942882145, 
                        876295483, 
                        774519349, 
                        1031303283, 
                        893073459, 
                        976565307, 
                        1937255978, 
                        859661936, 
                        993344312, 
                        3814708, 
                        65, 
                        0, 
                        37049520, 
                        0, 
                        37049632, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        4294901760, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0, 
                        0
                    ], 
                    "cwd": 0, 
                    "mxcsr_mask": 65535
                }, 
                "clear_tid_addr": 0, 
                "gpregs": {
                    "gs": "0x0", 
                    "ip": "0x7f172cf1ea04", 
                    "cx": "0xffffffffffffffff", 
                    "cs": "0x33", 
                    "ax": "0x38", 
                    "orig_ax": "0x38", 
                    "di": "0x1200011", 
                    "es": "0x0", 
                    "gs_base": "0x0", 
                    "r14": "0x0", 
                    "r15": "0x2355e00", 
                    "r12": "0x7ffffdbf74f0", 
                    "r13": "0x0", 
                    "r10": "0x7f172d83d9d0", 
                    "r11": "0x246", 
                    "fs_base": "0x7f172d83d700", 
                    "bp": "0x7ffffdbf7530", 
                    "dx": "0x0", 
                    "bx": "0x0", 
                    "ds": "0x0", 
                    "ss": "0x2b", 
                    "sp": "0x7ffffdbf74f0", 
                    "r8": "0x0", 
                    "r9": "0x0", 
                    "fs": "0x0", 
                    "si": "0x0", 
                    "flags": "0x246"
                }
            }, 
            "tc": {
                "timers": {
                    "real": {
                        "isec": 0, 
                        "vusec": 0, 
                        "iusec": 0, 
                        "vsec": 0
                    }, 
                    "virt": {
                        "isec": 0, 
                        "vusec": 0, 
                        "iusec": 0, 
                        "vsec": 0
                    }, 
                    "prof": {
                        "isec": 0, 
                        "vusec": 0, 
                        "iusec": 0, 
                        "vsec": 0
                    }
                }, 
                "cg_set": 1, 
                "signals_s": {}, 
                "blk_sigset": "0x10002", 
                "exit_code": 0, 
                "rlimits": {
                    "rlimits": [
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 8388608
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 0
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 62844, 
                            "cur": 62844
                        }, 
                        {
                            "max": 4096, 
                            "cur": 1024
                        }, 
                        {
                            "max": 65536, 
                            "cur": 65536
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }, 
                        {
                            "max": 62844, 
                            "cur": 62844
                        }, 
                        {
                            "max": 819200, 
                            "cur": 819200
                        }, 
                        {
                            "max": 0, 
                            "cur": 0
                        }, 
                        {
                            "max": 0, 
                            "cur": 0
                        }, 
                        {
                            "max": 18446744073709551615, 
                            "cur": 18446744073709551615
                        }
                    ]
                }, 
                "comm": "loop.sh", 
                "flags": 1077960704, 
                "task_state": 1, 
                "personality": 0
            }
        }
    ]
}

Generate core files out of task images

Moved into separate project called criu-coredump.

Show images statistics

E.g. -- total number of processes, files, memory, sockets, etc. Use 

crit x $directory_with_images <explore type>

Supported explorers are

'ps'
to show process tree
'fds'
to show files used by tasks
'mems'
to show memory mappings info

Planned functionality

On-the-fly conversion

There's an idea to make CRIU spawn CRIT and read images "through" it, to allow for at-the-restore-time modifications. For details, see https://github.com/xemul/criu/issues/59.

Convert between different image versions

Right now we store the images version in inventory.img and collect info about what's bad with V1 images. If some day we have v2, CRIT will convert from v1. And for backward compatibility we'll use on-the-fly conversion when restoring from old images.

Check/validate images

Check that

  • all images are present
  • the inter-images IDs are in consistent state

For details, see https://github.com/xemul/criu/issues/44.

Anonymize image files

Since images contain raw memory dumps people may refuse to send us images for debugging. Need to anonymize them, i.e. -- remove this sensitive information.

Retrieved from "https://criu.org/index.php?title=CRIT&oldid=4870"