Difference between revisions of "Iterative migration"

m
m (Add kill command)
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability to perform [http://en.wikipedia.org/wiki/Live_migration#Pre-copy_memory_migration pre-copy memory migration].
+
This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability to perform [[wikipedia:Live_migration#Pre-copy_memory_migration|pre-copy memory migration]].
  
 
{{Note|It is assumed that you already read [[Live migration]] article before this one.}}
 
{{Note|It is assumed that you already read [[Live migration]] article before this one.}}
Line 22: Line 22:
  
 
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 
  [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
   --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped
+
   --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem
  
 
Note that:
 
Note that:
Line 33: Line 33:
  
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 
  [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
 +
 +
or using rsync
 +
 +
[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>
 +
 +
Note: It is important to enable "copy symlinks as symlinks" (<code>-l</code> for <code>rsync</code>) to ensure that symlinks are preserved between previous directories. When <code>-r</code> is passed to <code>scp</code> it will follow symbolic links encountered in the tree traversal.
  
 
=== Restore ===
 
=== Restore ===
 
On the destination node restore the apps from images:
 
On the destination node restore the apps from images:
  
  [dst]# criu restore --tree <pid> --images-dir <path-to-images>
+
  [dst]# criu restore --images-dir <path-to-images>
  
 
=== Kill ===
 
=== Kill ===
 
If everything went OK you can kill stopped tasks on the source node:
 
If everything went OK you can kill stopped tasks on the source node:
  
  [src]# FIXME put command here
+
  [src]# kill <pid>
 +
 
 +
== Further reading ==
 +
 
 +
* [[P.Haul]] is the Go extension to CRIU doing [[live migration]]
  
 
[[Category: HOWTO]]
 
[[Category: HOWTO]]
 
[[Category: Memory]]
 
[[Category: Memory]]
 +
[[Category: Live migration]]

Latest revision as of 18:10, 12 January 2023

This page describes how to reduce the freeze time of an application by using the memory changes tracking ability to perform pre-copy memory migration.

Note.svg Note: It is assumed that you already read Live migration article before this one.

Migration sequenceEdit

The steps below look like those in regular live migration, but include one or more pre-dump stages.

Pre-dumpEdit

Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.

[src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>

The directory with images can be on a shared storage, or you can use disk-less migration to avoid the Copy step.

Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.

Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the --images-dir and the --prev-images-dir for all pre-dump and dump steps.

DumpEdit

Now you can do regular dump of your processes.

[src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \
 --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem

Note that:

  1. this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
  2. the --prev-images-dir should contain path to the directory with pre-dump images relative to the directory where the dump images will be put.

CopyEdit

Copy images to the destination node:

[src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>

or using rsync

[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>

Note: It is important to enable "copy symlinks as symlinks" (-l for rsync) to ensure that symlinks are preserved between previous directories. When -r is passed to scp it will follow symbolic links encountered in the tree traversal.

RestoreEdit

On the destination node restore the apps from images:

[dst]# criu restore --images-dir <path-to-images>

KillEdit

If everything went OK you can kill stopped tasks on the source node:

[src]# kill <pid>

Further readingEdit