Difference between revisions of "Iterative migration"

From CRIU
Jump to navigation Jump to search
(use {{Note}} to ask to read Live migration)
m (Add kill command)
 
(11 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability.
+
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability to perform [[wikipedia:Live_migration#Pre-copy_memory_migration|pre-copy memory migration]].
  
 
{{Note|It is assumed that you already read [[Live migration]] article before this one.}}
 
{{Note|It is assumed that you already read [[Live migration]] article before this one.}}
Line 5: Line 5:
 
== Migration sequence ==
 
== Migration sequence ==
  
The steps below look like those in regular live migration, but include one of more pre-dump stages.
+
The steps below look like those in regular live migration, but include one or more pre-dump stages.
  
 
=== Pre-dump ===
 
=== Pre-dump ===
Take tasks you're about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.
+
Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.
  
 
  [src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>
 
  [src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>
Line 14: Line 14:
 
The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step.
 
The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step.
  
Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory, that has changed from previous step. Doing several pre-dump stages (or iterations) may reduce the amount of data dumped on dump stage and thus -- smaller freeze time.
+
Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.
 +
 
 +
Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the <code>--images-dir</code> and the <code>--prev-images-dir</code> for all pre-dump and dump steps.
  
 
=== Dump ===
 
=== Dump ===
 
Now you can do regular dump of your processes.
 
Now you can do regular dump of your processes.
  
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped
+
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 +
  --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem
  
Note, that
+
Note that:
  
# this dump would work faster, than it would if there were no the pre-dump stage, as this dump only takes the memory that has changed, since the pre-dump step.
+
# this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
# the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put''
+
# the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put''.
  
 
=== Copy ===
 
=== Copy ===
Copy images to destination node:
+
Copy images to the destination node:
  
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 +
 +
or using rsync
 +
 +
[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>
 +
 +
Note: It is important to enable "copy symlinks as symlinks" (<code>-l</code> for <code>rsync</code>) to ensure that symlinks are preserved between previous directories. When <code>-r</code> is passed to <code>scp</code> it will follow symbolic links encountered in the tree traversal.
  
 
=== Restore ===
 
=== Restore ===
Go to the destination node and restore the apps from images on it:
+
On the destination node restore the apps from images:
  
  [dst]# criu restore --tree <pid> --images-dir <path-to-images>
+
  [dst]# criu restore --images-dir <path-to-images>
  
 
=== Kill ===
 
=== Kill ===
If everything went OK you can return on the source node and kill stopped tasks on it.
+
If everything went OK you can kill stopped tasks on the source node:
 +
 
 +
[src]# kill <pid>
 +
 
 +
== Further reading ==
  
[src]# FIXME put command here
+
* [[P.Haul]] is the Go extension to CRIU doing [[live migration]]
  
 
[[Category: HOWTO]]
 
[[Category: HOWTO]]
 +
[[Category: Memory]]
 +
[[Category: Live migration]]

Latest revision as of 18:10, 12 January 2023

This page describes how to reduce the freeze time of an application by using the memory changes tracking ability to perform pre-copy memory migration.

Note.svg Note: It is assumed that you already read Live migration article before this one.

Migration sequence[edit]

The steps below look like those in regular live migration, but include one or more pre-dump stages.

Pre-dump[edit]

Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.

[src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>

The directory with images can be on a shared storage, or you can use disk-less migration to avoid the Copy step.

Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.

Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the --images-dir and the --prev-images-dir for all pre-dump and dump steps.

Dump[edit]

Now you can do regular dump of your processes.

[src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem

Note that:

  1. this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
  2. the --prev-images-dir should contain path to the directory with pre-dump images relative to the directory where the dump images will be put.

Copy[edit]

Copy images to the destination node:

[src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>

or using rsync

[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>

Note: It is important to enable "copy symlinks as symlinks" (-l for rsync) to ensure that symlinks are preserved between previous directories. When -r is passed to scp it will follow symbolic links encountered in the tree traversal.

Restore[edit]

On the destination node restore the apps from images:

[dst]# criu restore --images-dir <path-to-images>

Kill[edit]

If everything went OK you can kill stopped tasks on the source node:

[src]# kill <pid>

Further reading[edit]