Difference between revisions of "Iterative migration"
(use {{Note}} to ask to read Live migration) |
Prajwal S N (talk | contribs) m (Add kill command) |
||
| (11 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| − | This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability. | + | This page describes how to reduce the freeze time of an application by using the [[memory changes tracking]] ability to perform [[wikipedia:Live_migration#Pre-copy_memory_migration|pre-copy memory migration]]. |
{{Note|It is assumed that you already read [[Live migration]] article before this one.}} | {{Note|It is assumed that you already read [[Live migration]] article before this one.}} | ||
| Line 5: | Line 5: | ||
== Migration sequence == | == Migration sequence == | ||
| − | The steps below look like those in regular live migration, but include one | + | The steps below look like those in regular live migration, but include one or more pre-dump stages. |
=== Pre-dump === | === Pre-dump === | ||
| − | Take tasks you | + | Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump. |
[src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A> | [src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A> | ||
| Line 14: | Line 14: | ||
The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step. | The directory with images can be on a shared storage, or you can use [[disk-less migration]] to avoid the '''Copy''' step. | ||
| − | Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory | + | Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time. |
| + | |||
| + | Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the <code>--images-dir</code> and the <code>--prev-images-dir</code> for all pre-dump and dump steps. | ||
=== Dump === | === Dump === | ||
Now you can do regular dump of your processes. | Now you can do regular dump of your processes. | ||
| − | [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped | + | [src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \ |
| + | --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem | ||
| − | Note | + | Note that: |
| − | # this dump would work faster | + | # this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump; |
| − | # the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put'' | + | # the <code>--prev-images-dir</code> should contain path to the directory with pre-dump images ''relative to the directory where the dump images will be put''. |
=== Copy === | === Copy === | ||
| − | Copy images to destination node: | + | Copy images to the destination node: |
[src]# scp -r <path-to-images-dir> <dst>:/<path-to-images> | [src]# scp -r <path-to-images-dir> <dst>:/<path-to-images> | ||
| + | |||
| + | or using rsync | ||
| + | |||
| + | [src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images> | ||
| + | |||
| + | Note: It is important to enable "copy symlinks as symlinks" (<code>-l</code> for <code>rsync</code>) to ensure that symlinks are preserved between previous directories. When <code>-r</code> is passed to <code>scp</code> it will follow symbolic links encountered in the tree traversal. | ||
=== Restore === | === Restore === | ||
| − | + | On the destination node restore the apps from images: | |
| − | [dst]# criu restore | + | [dst]# criu restore --images-dir <path-to-images> |
=== Kill === | === Kill === | ||
| − | If everything went OK you can | + | If everything went OK you can kill stopped tasks on the source node: |
| + | |||
| + | [src]# kill <pid> | ||
| + | |||
| + | == Further reading == | ||
| − | + | * [[P.Haul]] is the Go extension to CRIU doing [[live migration]] | |
[[Category: HOWTO]] | [[Category: HOWTO]] | ||
| + | [[Category: Memory]] | ||
| + | [[Category: Live migration]] | ||
Latest revision as of 18:10, 12 January 2023
This page describes how to reduce the freeze time of an application by using the memory changes tracking ability to perform pre-copy memory migration.
|
Note: It is assumed that you already read Live migration article before this one. |
Migration sequence[edit]
The steps below look like those in regular live migration, but include one or more pre-dump stages.
Pre-dump[edit]
Take tasks you are about to migrate and pre-dump them into some place. Tasks will remain running after pre-dump, unlike regular dump.
[src]# criu pre-dump --tree <pid> --images-dir <path-to-existing-directory-A>
The directory with images can be on a shared storage, or you can use disk-less migration to avoid the Copy step.
Now you can either proceed to next step and do regular dump, or perform the pre-dump step again. In the latter case pre-dump would generate another set of pre-dump images which will contain memory changed after previous pre-dump. Doing several pre-dump iterations may reduce the amount of data dumped on dump stage and thus lead to shorter freeze time.
Note, that if you're going to perform more than one pre-dump steps, you should create different directories for images and properly reference them with the --images-dir and the --prev-images-dir for all pre-dump and dump steps.
Dump[edit]
Now you can do regular dump of your processes.
[src]# criu dump --tree <pid> --images-dir <path-to-existing-directory-B> \ --prev-images-dir <path-to-directory-A-relative-to-B> --leave-stopped --track-mem
Note that:
- this dump would work faster than without pre-dump, as this dump only takes the memory that has changed since the last pre-dump;
- the
--prev-images-dirshould contain path to the directory with pre-dump images relative to the directory where the dump images will be put.
Copy[edit]
Copy images to the destination node:
[src]# scp -r <path-to-images-dir> <dst>:/<path-to-images>
or using rsync
[src]# rsync -rl <path-to-images-dir>/ <dst>:<path-to-images>
Note: It is important to enable "copy symlinks as symlinks" (-l for rsync) to ensure that symlinks are preserved between previous directories. When -r is passed to scp it will follow symbolic links encountered in the tree traversal.
Restore[edit]
On the destination node restore the apps from images:
[dst]# criu restore --images-dir <path-to-images>
Kill[edit]
If everything went OK you can kill stopped tasks on the source node:
[src]# kill <pid>
Further reading[edit]
- P.Haul is the Go extension to CRIU doing live migration