Checkpoint/Restore - Revision history

OpenGridSchedulerGridEngine: /* Switch to restorer context, restore the rest and continue */ spelling

2017-05-12T11:05:50Z

Switch to restorer context, restore the rest and continue: spelling

Kir: /* Restore basic tasks resources */ typos

2017-02-23T02:43:43Z

Restore basic tasks resources: typos

Kir: rm links from headers, some english fixes

2016-09-19T23:44:14Z

rm links from headers, some english fixes

Xemul: /* Switch to restorer context, restore the rest and continue */

2016-09-17T16:18:18Z

Switch to restorer context, restore the rest and continue

Xemul: /* Collect process tree and freeze it */

2016-09-17T16:17:22Z

Collect process tree and freeze it

Xemul: /* Restore basic tasks resources */

2016-09-17T14:29:01Z

Restore basic tasks resources

Artem: /* Checkpoint */

2015-08-05T06:25:57Z

Checkpoint

Xemul: /* Switch to restorer context, restore the rest and continue */

2014-12-01T12:00:40Z

Switch to restorer context, restore the rest and continue

Xemul: /* Collect tasks' resources and dump them */

2014-12-01T12:00:24Z

Collect tasks' resources and dump them

Xemul at 11:15, 1 December 2014

2014-12-01T11:15:17Z

@@ Line 61: / Line 61: @@
 The reason for restorer blob is simple. Since criu morphs into the target process, it will have to unmap all its memory and put back the target one. While doing so, some code should exist in memory (the code doing the munmap and mmap). Therefore, the restorer blob is introduced. It's a small piece of code, that doesn't intersect with criu mappings AND target mappings. At the end of stage 2 criu jumps into this blob and restores the memory maps.
-At the same place we restore timers not to make them fire too early, here we restore credentials to let criu do priveledged operations (like fork-with-pid) and threads not to make them suffer from sudden memory layout change.
+At the same place we restore timers not to make them fire too early, here we restore credentials to let criu do privileged operations (like fork-with-pid) and threads not to make them suffer from sudden memory layout change.
 ''See also: [[restorer context]], [[tree after restore]].''
 [[Category:Under the hood]]

@@ Line 55: / Line 55: @@
 # threads
-These for are delayed till the last stage for the reasons described further. On this stage CRIU opens files, prepares [[namespaces]], maps (and fills with data) private memory areas, creates sockets, calls chdir() and chroot() and dome some more.
+The restoration of the above four types of resources are delayed till the last stage for the reasons described below. On this stage CRIU opens files, prepares [[namespaces]], maps (and fills with data) private memory areas, creates sockets, calls chdir() and chroot() and doing some more.
 === Switch to restorer context, restore the rest and continue ===

@@ Line 4: / Line 4: @@
 The checkpoint procedure relies heavily on '''/proc''' file system (it's a general place where criu takes all the information it needs).
-Which includes
+The information gathered from /proc includes:
 * Files descriptors information (via '''/proc/$pid/fd''' and '''/proc/$pid/fdinfo''').
@@ Line 11: / Line 11: @@
 * etc.
-The process dumper (lets call it a dumper further) does the following steps during checkpoint stage
+The process dumper (called a dumper below) does the following steps during the checkpoint stage.
 === Collect process tree and freeze it ===
@@ Line 57: / Line 57: @@
 These for are delayed till the last stage for the reasons described further. On this stage CRIU opens files, prepares [[namespaces]], maps (and fills with data) private memory areas, creates sockets, calls chdir() and chroot() and dome some more.
-=== Switch to [[restorer context]], restore the rest and continue ===
+=== Switch to restorer context, restore the rest and continue ===
-The reason for restorer blob is simple. Since criu morphs into the target process, it will have to unmap all its memory and put back the target one. While doing so some code should exist in memory (the code doing the munmap and mmap). So we introduced the restorer blob -- the small piece of code, that doesn't intersect with criu mappings AND target mappings. At the end of stage 2 criu jumps into this blob and restores the memory maps.
+The reason for restorer blob is simple. Since criu morphs into the target process, it will have to unmap all its memory and put back the target one. While doing so, some code should exist in memory (the code doing the munmap and mmap). Therefore, the restorer blob is introduced. It's a small piece of code, that doesn't intersect with criu mappings AND target mappings. At the end of stage 2 criu jumps into this blob and restores the memory maps.
 At the same place we restore timers not to make them fire too early, here we restore credentials to let criu do priveledged operations (like fork-with-pid) and threads not to make them suffer from sudden memory layout change.
-''See also: [[Tree after restore]]''
+''See also: [[restorer context]], [[tree after restore]].''
 [[Category:Under the hood]]

← Older revision		Revision as of 16:18, 17 September 2016
Line 62:		Line 62:

	At the same place we restore timers not to make them fire too early, here we restore credentials to let criu do priveledged operations (like fork-with-pid) and threads not to make them suffer from sudden memory layout change.		At the same place we restore timers not to make them fire too early, here we restore credentials to let criu do priveledged operations (like fork-with-pid) and threads not to make them suffer from sudden memory layout change.
		+
		+	''See also: [[Tree after restore]]''

	[[Category:Under the hood]]		[[Category:Under the hood]]

@@ Line 15: / Line 15: @@
 === Collect process tree and freeze it ===
 The '''$pid''' of a process group leader is obtained from the command line (<code>--tree</code> option). By using this '''$pid''' the dumper walks though '''/proc/$pid/task/''' directory collecting threads and through the '''/proc/$pid/task/$tid/children''' to gathers children recursively. While walking tasks are stopped using the <code>ptrace</code>'s <code>PTRACE_SEIZE</code> command.
 === Collect tasks' resources and dump them ===

@@ Line 22: / Line 22: @@
 # Core parameters of a task (such as registers and friends) are being dumped via ptrace interface and parsing '''/proc/$pid/stat''' entry.
-Then CRIU injects a parasite code into a task via ptrace interface. This is done in two steps -- at first we inject only a few bytes for ''mmap'' syscall at CS:IP the task has at moment of seizing. Then ptrace allow us to run an injected syscall and we allocate enough memory for a parasite code chunk we need for dumping. After that the parasite code is copied into new place inside dumpee address space and CS:IP set respectively to point to our parasite code.
+Then CRIU injects a [[parasite code]] into a task via ptrace interface. This is done in two steps -- at first we inject only a few bytes for ''mmap'' syscall at CS:IP the task has at moment of seizing. Then ptrace allow us to run an injected syscall and we allocate enough memory for a parasite code chunk we need for dumping. After that the parasite code is copied into new place inside dumpee address space and CS:IP set respectively to point to our parasite code.
 From parsite context CRIU does more information such as
 # Credentials
 # Contents of memory
 === Cleanup ===

@@ Line 1: / Line 1: @@
-== Basic design ==
+This page describes the overall design of how Checkpoint and Restore work in CRIU.
-=== Checkpoint ===
+== Checkpoint ==
 The checkpoint procedure relies heavily on '''/proc''' file system (it's a general place where criu takes all the information it needs).
@@ Line 13: / Line 13: @@
 The process dumper (lets call it a dumper further) does the following steps during checkpoint stage
-==== Collect process tree and freeze it ====
+=== Collect process tree and freeze it ===
 The '''$pid''' of a process group leader is obtained from the command line (<code>--tree</code> option). By using this '''$pid''' the dumper walks though '''/proc/$pid/task/''' directory collecting threads and through the '''/proc/$pid/task/$tid/children''' to gathers children recursively. While walking tasks are stopped using the <code>ptrace</code>'s <code>PTRACE_SEISE</code> command.
-==== Collect tasks' resources and dump them ====
+=== Collect tasks' resources and dump them ===
 At this step CRIU reads all the information (it knows) about collected tasks and writes them to dump files. The resources are obtained via
 # VMAs areas are parsed from '''/proc/$pid/smaps''' and mapped files are read from '''/proc/$pid/map_files''' links
@@ Line 29: / Line 29: @@
-==== Cleanup ====
+=== Cleanup ===
 After everything dumped (such as memory pages, which can be written out only from inside dumpee address space) we use ptrace facility again and cure dumpee by dropping out all our parasite code and restoring original code. Then CRIU detaches from tasks and they continue to operate.
-=== Restore ===
+== Restore ==
-The restore procedure (aka restorer) proceed in the following steps
+The restore procedure (aka restorer) is done by CRIU morphing itself into the tasks it restores. On the top-level it consists of 4 steps
-# A process tree has been read from a file.
+=== Resolve shared resources ===
-# Every process started with saved (i.e. original) '''$pid''' (see [[Pid restore]]) via <code>clone()</code> call.
-# Files and pipes are restored (by restored it's meant - they are opened and positioned).
+At this step CRIU reads in image files and finds out which processes share which resources. Later shared resources are restored by some one process and all the others either inherit one on the 2nd stage (like session) or obtain in some other way. The latter is, for example, shared files which are sent with SCM_CREDS messages via unix sockets, or shared memory areas that are restoring via <code>memfd</code> file descriptor.
-# A new memory map is created, filled with data the program had at checkpoint time.
-# Finally the program is kicked to start with rt_sigreturn system call.
+=== Fork the process tree ===
 [[Category:Under the hood]]