79
edits
Changes
Jump to navigation
Jump to search
Line 8:
Line 8: − + Line 20:
Line 20: − + Line 40:
Line 40: − + Line 78:
Line 78: − + − + − + − + − + − We can try to use sys_read_process_vm() syscall to mitigate all of the above. To do this we+ − + − + + Line 92:
Line 93: + − + Line 102:
Line 104: − + Line 120:
Line 122: + Line 134:
Line 137: − + Line 141:
Line 144: + + Line 147:
Line 152: − − === Implement diskless migration === − − '''Summary:''' Need to investigate and implement that named diskless migration. − − By diskless we imply a case where all images generated by checkpoint procedure do not sit on storage at all but rather get collected by the criu service on a destination machine, and read from memory later once restore procedure is initiated. More importantly the memory transferred should be deduplicated on the fly and premapped at some preliminary address. Later the restore procedure just remap data to proper positions without copying page data at all. − − This task is under the question still and the section is like a placeholder. − − '''Details:''' − * Skill level: expert − * Language: C − * Mentor: Cyrill Gorcunov <gorcunov@gmail.com> − * Suggested by: Cyrill Gorcunov <gorcunov@gmail.com> Line 167:
Line 158: − + + − + Line 175:
Line 167: + Line 181:
Line 174: + + + + + + + + + + + + + + + + + + +
no edit summary
=== Post-copy for shared memory and hugetlbfs ===
=== Post-copy for shared memory and hugetlbfs ===
'''Summary:''' extend post-copy memory restore and migration to support shared memory and hugeltbfs.
'''Summary:''' extend post-copy memory restore and migration to support shared memory and hugetlbfs.
CRIU relies on [[Userfaultfd]] mechanism in the Linux kernel to implement the demand paging in userspace and allow post-copy memory (or lazy) [[Lazy_migration|migration]]. However, currently this support is limited to anonymous private memory mappings, although kernel also supports shared memory areas and hugetlbfs backed memory.
CRIU relies on [[Userfaultfd]] mechanism in the Linux kernel to implement the demand paging in userspace and allow post-copy memory (or lazy) [[Lazy_migration|migration]]. However, currently this support is limited to anonymous private memory mappings, although kernel also supports shared memory areas and hugetlbfs backed memory.
* Skill level: most probably advanced?
* Skill level: most probably advanced?
* Language: C
* Language: C
* Mentor: Mike Rapoport <rppt@linux.ibm.com>
* Mentor: Mike Rapoport <rppt@linux.ibm.com> / Andrey Vagin <avagin@gmail.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>
* Skill level: intermediate
* Skill level: intermediate
* Language: C, though decoder/preprocessor can be in any language
* Language: C, though decoder/preprocessor can be in any language
* Mentor: Andrei Vagin <avagin@gmail.com>, Pavel Emelyanov <xemul@openvz.org>
* Mentor: Andrei Vagin <avagin@gmail.com> / Pavel Emelyanov <xemul@openvz.org>
* Suggested by: Andrei Vagin <avagin@gmail.com>
* Suggested by: Andrei Vagin <avagin@gmail.com>
Current [[CLI/cmd/pre-dump|pre-dump]] mode is used to write task memory contents into image
Current [[CLI/cmd/pre-dump|pre-dump]] mode is used to write task memory contents into image
files w/o stopping the task for too long. It does this by stopping the task, infecting it and
files w/o stopping the task for too long. It does this by stopping the task, infecting it and
draining all the memory into a set of pipes. Then the task is cured and resumed and the pipes'
draining all the memory into a set of pipes. Then the task is cured, resumed and the pipes'
contents is written into images (maybe a [[page server]]). This approach creates a big stress
contents is written into images (maybe a [[page server]]). Unfortunately, this approach creates
on memory subsystem, as keeping all the memory in pipes creates a lot of unreclaimable memory
a big stress on the memory subsystem, as keeping all memory in pipes creates a lot of unreclaimable
(pages in pipes are not swappable), as well as the number of pipes themselves can be buge (as
memory (pages in pipes are not swappable), as well as the number of pipes themselves can be huge, as
one pipe doesn't store more than a fixed certain amount of data).
one pipe doesn't store more than a fixed amount of data (see pipe(7) man page).
A solution for this problem is to use a sys_read_process_vm() syscall, which will mitigate
need to allocate a temporary buffer in criu, then walk the target process vm by copying the
all of the above. To do this we need to allocate a temporary buffer in criu, then walk the
memory piece-by-piece into it, then flush the data into image (or page server), then repeat.
target process vm by copying the memory piece-by-piece into it, then flush the data into image
(or page server), and repeat.
Ideally there should be sys_splice_process_vm() syscall in the kernel, that does the same as
Ideally there should be sys_splice_process_vm() syscall in the kernel, that does the same as
'''Links:'''
'''Links:'''
* [[Memory pre dump]]
* https://github.com/checkpoint-restore/criu/issues/351
* https://github.com/checkpoint-restore/criu/issues/351
* [[Memory dumping and restoring]], [[Memory changes tracking]]
* [[Memory dumping and restoring]], [[Memory changes tracking]]
* [http://man7.org/linux/man-pages/man2/process_vm_readv.2.html process_vm_readv(2)] [http://man7.org/linux/man-pages/man2/vmsplice.2.html vmsplice(2)] [https://lkml.org/lkml/2017/11/22/527 RFC for splice_process_vm syscall]
* [http://man7.org/linux/man-pages/man2/process_vm_readv.2.html process_vm_readv(2)] [http://man7.org/linux/man-pages/man2/vmsplice.2.html vmsplice(2)] [https://lkml.org/lkml/2018/1/9/32 RFC for splice_process_vm syscall]
'''Details:'''
'''Details:'''
* Suggested by: Pavel Emelianov <xemul@virtuozzo.com>
* Suggested by: Pavel Emelianov <xemul@virtuozzo.com>
=== Anonymize image files ===
=== Anonymise image files ===
'''Summary:''' Teach [[CRIT]] to remove sensitive information from images
'''Summary:''' Teach [[CRIT]] to remove sensitive information from images
'''Links:'''
'''Links:'''
* [[Anonymize image files]]
* https://github.com/checkpoint-restore/criu/issues/360
* https://github.com/checkpoint-restore/criu/issues/360
* [[CRIT]], [[Images]]
* [[CRIT]], [[Images]]
'''Summary:''' Implement image view and manipulation in Go
'''Summary:''' Implement image view and manipulation in Go
CRIU's checkpoint images are stored on disk using protobuf. For easier analysis of checkpoint files CRIU has a tool called [[CRIT|CRiu Image Tool (CRIT)]]. It can display/decode CRIU image files from binary protobuf to JSON as well as encode JSON files back to the binary format. With closer integration of CRIU in container runtimes it becomes important to be able to view the CRIU output files. Either for manipulation before restoring or for reading checkpoint statistics (memory pages written to disk, memory page skipped, process downtime).
CRIU's checkpoint images are stored on disk using protobuf. For easier analysis of checkpoint files CRIU has a tool called [[CRIT|CRiu Image Tool (CRIT)]]. It can display/decode CRIU image files from binary protobuf to JSON as well as encode JSON files back to the binary format. With closer integration of CRIU in container runtimes it becomes important to be able to view the CRIU output files. Either for manipulation before restoring or for reading checkpoint statistics (memory pages written to disk, memory pages skipped, process downtime).
Currently CRIT is implemented in Python, for easier integration in other Go projects it is important to have image manipulation and analysis available from GO. This means we need a Go based library to read/modify/write/encode/decode CRIU's image files. Based on this library a Go based implementation of CRIT would be useful.
Currently CRIT is implemented in Python, for easier integration in other Go projects it is important to have image manipulation and analysis available from GO. This means we need a Go based library to read/modify/write/encode/decode CRIU's image files. Based on this library a Go based implementation of CRIT would be useful.
* [[CRIT]]
* [[CRIT]]
* Possible use case see LXD: https://github.com/lxc/lxd/blob/cb55b1c5a484a43e0c21c6ae8c4a2e30b4d45be3/lxd/migrate_container.go#L179
* Possible use case see LXD: https://github.com/lxc/lxd/blob/cb55b1c5a484a43e0c21c6ae8c4a2e30b4d45be3/lxd/migrate_container.go#L179
* https://github.com/lxc/lxd/pull/4072
* https://github.com/checkpoint-restore/go-criu/blob/master/phaul/stats.go
'''Details:'''
'''Details:'''
* Mentor: Adrian Reber <areber@redhat.com>
* Mentor: Adrian Reber <areber@redhat.com>
* Suggested by: Adrian Reber <areber@redhat.com>
* Suggested by: Adrian Reber <areber@redhat.com>
=== Memory changes tracking with userfaultfd-WP ===
=== Memory changes tracking with userfaultfd-WP ===
Currently CRIU uses [[Memory_changes_tracking|Soft-dirty]] mechanism in Linux kernel to track memory changes.
Currently CRIU uses [[Memory_changes_tracking|Soft-dirty]] mechanism in Linux kernel to track memory changes.
This mechanism can be complemented or even completely replaced with recently proposed userfaultfd-WP.
This mechanism can be complemented (or even completely replaced) with recently proposed write protection support for
userfaultfd (userfaultfd-WP).
Userfault allows implementation of paging in userspace. It allows an application to receive notifications about page faults and provide the desired memory contents for the faulting pages. In the current upstream kernels only missing page faults are supported, but there is an ongoing work to allow notifications for write faults as well. Using these notifications it would be possible to precisely track memory accesses of during pre-dump iterations and this approach may prove more efficient than soft-dirty.
Userfault allows implementation of paging in userspace. It allows an application to receive notifications about page faults and provide the desired memory contents for the faulting pages. In the current upstream kernels only missing page faults are supported, but there is an ongoing work to allow notifications for write faults as well. Using such notifications it would be possible to precisely track memory changes during pre-dump iterations. This approach may prove to be more efficient than soft-dirty.
'''Links:'''
'''Links:'''
* [https://github.com/xzpeter/linux/tree/uffd-wp-merged Userfaultfd-WP]
* [https://github.com/xzpeter/linux/tree/uffd-wp-merged Userfaultfd-WP]
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/soft-dirty.html?highlight=soft%20dirty Soft-Dirty]
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/soft-dirty.html?highlight=soft%20dirty Soft-Dirty]
* https://lwn.net/Articles/777258/
'''Details:'''
'''Details:'''
* Mentor: Mike Rapoport <rppt@linux.ibm.com>
* Mentor: Mike Rapoport <rppt@linux.ibm.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>
=== Use eBPF to lock and unlock the network ===
'''Summary:''' Use ePBF instead of external iptables-restore tool for network lock and unlock.
During checkpointing and restoring CRIU locks the network to make sure no network packets are accepted by the network stack during the time the process is checkpointed. Currently CRIU calls out to iptables-restore to create and delete the corresponding iptables rules. Another approach which avoids calling out to the external binary iptables-restore would be to directly inject eBPF rules. There have been reports from users that iptables-restore fails in some way and eBPF could avoid this external dependency.
'''Links:'''
* https://www.criu.org/TCP_connection#Checkpoint_and_restore_TCP_connection
* https://github.com/systemd/systemd/blob/master/src/core/bpf-firewall.c
'''Details:'''
* Skill level: intermediate
* Language: C
* Mentor: Adrian Reber <areber@redhat.com>
* Suggested by: Adrian Reber <areber@redhat.com>
[[Category:GSoC]]
[[Category:Development]]