Difference between revisions of "Iterative migration"

From CRIU
Jump to navigation Jump to search
(Created page with "Please, read the article about Live migration before this one. This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]...")
 
m (Add kill command)
 
(13 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Please, read the article about [[Live migration]] before this one.
+
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability to perform [[wikipedia:Live_migration#Pre-copy_memory_migration|pre-copy memory migration]].
  
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability.
+
{{Note|It is assumed that you already read [[Live migration]] article before this one.}}
  
 
== Migration sequence ==
 
== Migration sequence ==
  
The steps below look like those in regular live migration, but include one of more pre-dump stages.
+
The steps below look like those in regular live migration, but include one or more pre-dump stages.
  
 
=== Pre-dump ===
 
=== Pre-dump ===
Take tasks you're about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.
+
Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.
  
 
  [src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>
 
  [src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>
  
 
The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step.
 
The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step.
 +
 +
Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.
 +
 +
Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the <code>--images-dir</code> and the <code>--prev-images-dir</code> for all pre-dump and dump steps.
  
 
=== Dump ===
 
=== Dump ===
 
Now you can do regular dump of your processes.
 
Now you can do regular dump of your processes.
  
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped
+
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 +
  --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem
  
Note, that
+
Note that:
  
# this dump would work faster, than it would if there were no the pre-dump stage, as this dump only takes the memory that has changed, since the pre-dump step.
+
# this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
# the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put''
+
# the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put''.
  
 
=== Copy ===
 
=== Copy ===
Copy images to destination node:
+
Copy images to the destination node:
  
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 +
 +
or using rsync
 +
 +
[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>
 +
 +
Note: It is important to enable "copy symlinks as symlinks" (<code>-l</code> for <code>rsync</code>) to ensure that symlinks are preserved between previous directories. When <code>-r</code> is passed to <code>scp</code> it will follow symbolic links encountered in the tree traversal.
  
 
=== Restore ===
 
=== Restore ===
Go to the destination node and restore the apps from images on it:
+
On the destination node restore the apps from images:
  
  [dst]# criu restore --tree <pid> --images-dir <path-to-images>
+
  [dst]# criu restore --images-dir <path-to-images>
  
 
=== Kill ===
 
=== Kill ===
If everything went OK you can return on the source node and kill stopped tasks on it.
+
If everything went OK you can kill stopped tasks on the source node:
 +
 
 +
[src]# kill <pid>
 +
 
 +
== Further reading ==
  
[src]# FIXME put command here
+
* [[P.Haul]] is the Go extension to CRIU doing [[live migration]]
  
 
[[Category: HOWTO]]
 
[[Category: HOWTO]]
 +
[[Category: Memory]]
 +
[[Category: Live migration]]

Latest revision as of 18:10, 12 January 2023

This page describes how to reduce the freeze time of an application by using the memory changes tracking ability to perform pre-copy memory migration.

Note.svg Note: It is assumed that you already read Live migration article before this one.

Migration sequence[edit]

The steps below look like those in regular live migration, but include one or more pre-dump stages.

Pre-dump[edit]

Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.

[src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>

The directory with images can be on a shared storage, or you can use disk-less migration to avoid the Copy step.

Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.

Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the --images-dir and the --prev-images-dir for all pre-dump and dump steps.

Dump[edit]

Now you can do regular dump of your processes.

[src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem

Note that:

  1. this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
  2. the --prev-images-dir should contain path to the directory with pre-dump images relative to the directory where the dump images will be put.

Copy[edit]

Copy images to the destination node:

[src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>

or using rsync

[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>

Note: It is important to enable "copy symlinks as symlinks" (-l for rsync) to ensure that symlinks are preserved between previous directories. When -r is passed to scp it will follow symbolic links encountered in the tree traversal.

Restore[edit]

On the destination node restore the apps from images:

[dst]# criu restore --images-dir <path-to-images>

Kill[edit]

If everything went OK you can kill stopped tasks on the source node:

[src]# kill <pid>

Further reading[edit]