Difference between revisions of "GSoC completed projects"
(Created page with "Category:GSoC === Restrict checks for open/mmaped files === '''Summary:''' Make sure the file opened (for fd or mapping) at restore is "the same" as it was on dump '''...") |
|||
| Line 5: | Line 5: | ||
'''Summary:''' Make sure the file opened (for fd or mapping) at restore is "the same" as it was on dump | '''Summary:''' Make sure the file opened (for fd or mapping) at restore is "the same" as it was on dump | ||
| − | '''Merged | + | '''Merged:''' https://github.com/checkpoint-restore/criu/pull/1123 |
CRIU doesn't carry files contents (except for ghost ones) into images. Thus on dump it saves some "meta" for file to validate it's "the same" on restore. Currently this meta includes only the file size. The task is to add some cookie value that's somehow affected by file's contents. This is primarily needed to reduce the possibility to restore with wrong libraries. | CRIU doesn't carry files contents (except for ghost ones) into images. Thus on dump it saves some "meta" for file to validate it's "the same" on restore. Currently this meta includes only the file size. The task is to add some cookie value that's somehow affected by file's contents. This is primarily needed to reduce the possibility to restore with wrong libraries. | ||
| Line 12: | Line 12: | ||
* https://www.criu.org/Category:Files | * https://www.criu.org/Category:Files | ||
* https://en.wikipedia.org/wiki/File_verification | * https://en.wikipedia.org/wiki/File_verification | ||
| + | |||
| + | === Optimize the pre-dump algorithm === | ||
| + | |||
| + | '''Summary:''' Optimize the pre-dump algorithm to avoid pinning to many memory in RAM | ||
| + | |||
| + | '''Merged:''' https://github.com/checkpoint-restore/criu/commit/98608b90de0f853b1c8a6e15b312320e1441c359 | ||
| + | |||
| + | Current [[CLI/cmd/pre-dump|pre-dump]] mode is used to write task memory contents into image | ||
| + | files w/o stopping the task for too long. It does this by stopping the task, infecting it and | ||
| + | draining all the memory into a set of pipes. Then the task is cured, resumed and the pipes' | ||
| + | contents is written into images (maybe a [[page server]]). Unfortunately, this approach creates | ||
| + | a big stress on the memory subsystem, as keeping all memory in pipes creates a lot of unreclaimable | ||
| + | memory (pages in pipes are not swappable), as well as the number of pipes themselves can be huge, as | ||
| + | one pipe doesn't store more than a fixed amount of data (see pipe(7) man page). | ||
| + | |||
| + | A solution for this problem is to use a sys_read_process_vm() syscall, which will mitigate | ||
| + | all of the above. To do this we need to allocate a temporary buffer in criu, then walk the | ||
| + | target process vm by copying the memory piece-by-piece into it, then flush the data into image | ||
| + | (or page server), and repeat. | ||
| + | |||
| + | Ideally there should be sys_splice_process_vm() syscall in the kernel, that does the same as | ||
| + | the read_process_vm does, but vmsplices the data | ||
| + | |||
| + | '''Links:''' | ||
| + | * [[Memory pre dump]] | ||
| + | * https://github.com/checkpoint-restore/criu/issues/351 | ||
| + | * [[Memory dumping and restoring]], [[Memory changes tracking]] | ||
| + | * [http://man7.org/linux/man-pages/man2/process_vm_readv.2.html process_vm_readv(2)] [http://man7.org/linux/man-pages/man2/vmsplice.2.html vmsplice(2)] [https://lkml.org/lkml/2018/1/9/32 RFC for splice_process_vm syscall] | ||
Revision as of 16:24, 9 February 2021
Restrict checks for open/mmaped files
Summary: Make sure the file opened (for fd or mapping) at restore is "the same" as it was on dump
Merged: https://github.com/checkpoint-restore/criu/pull/1123
CRIU doesn't carry files contents (except for ghost ones) into images. Thus on dump it saves some "meta" for file to validate it's "the same" on restore. Currently this meta includes only the file size. The task is to add some cookie value that's somehow affected by file's contents. This is primarily needed to reduce the possibility to restore with wrong libraries.
Links:
Optimize the pre-dump algorithm
Summary: Optimize the pre-dump algorithm to avoid pinning to many memory in RAM
Merged: https://github.com/checkpoint-restore/criu/commit/98608b90de0f853b1c8a6e15b312320e1441c359
Current pre-dump mode is used to write task memory contents into image files w/o stopping the task for too long. It does this by stopping the task, infecting it and draining all the memory into a set of pipes. Then the task is cured, resumed and the pipes' contents is written into images (maybe a page server). Unfortunately, this approach creates a big stress on the memory subsystem, as keeping all memory in pipes creates a lot of unreclaimable memory (pages in pipes are not swappable), as well as the number of pipes themselves can be huge, as one pipe doesn't store more than a fixed amount of data (see pipe(7) man page).
A solution for this problem is to use a sys_read_process_vm() syscall, which will mitigate all of the above. To do this we need to allocate a temporary buffer in criu, then walk the target process vm by copying the memory piece-by-piece into it, then flush the data into image (or page server), and repeat.
Ideally there should be sys_splice_process_vm() syscall in the kernel, that does the same as the read_process_vm does, but vmsplices the data
Links: