Collected on this page is the design notes about supporting userfaultfd in CRIU
Concepts
- Only MAP_PRIVATE | MAP_ANONYMOUS will be supported in the 1st version due to kernel constraints
- The
restoreaction should accept yet another API switch: option--lazy-pages
Daemon
Tasks after restore should have lazy VMAs being backed by userfaultfd, the fd itself should be sent before resume to CRIU (daemon?) and closed. This is CRIU who will monitor the UFFD events and repopulate the tasks address space. It should be able to get pages from both -- remote and local images.
Local images
The daemon should just use local page-read engine and read pages from images.
Remote images
- The page-read engine should be patched to learn how to talk to the remote host (page server with --page-server option?) on the other end.
- The source node should get pages from tasks dumped and send them out on the destination node.
- Protocol should include out-of-order pages and background pages pushing (sending them before demand from the process).
Known issues
Userfault is known not to map one page into two places. Thus -- COW-ed pages will get COW-ed.