32bit tasks C/R

Compatible applications

On x86_64 there are two types of compatible applications:

• ia32 - compiled to run on i686 target, can be executed on x86_64 with IA32_EMULATION config option set.
• x32 - specially compiled binaries to run on x86_64 machine with CONFIG_X86_X32 config option set.

Both of them uses 4 byte pointers thus can address no more than 4Gb of virtual memory.
But x32 uses full 64-bit register set (and thus can't be launched on i686 host natively).
Both of them requires additional environment on x86_64 as Glibc, libraries, and compiler support.
x32 is rarely distributed (at this moment only Debian x32 port can be easily found).
So, CRIU will support ia32 C/R at this moment, x32 support may be quite easily added on top of ia32 as needed patches have already added in kernel with ia32 C/R support.
The following text uses compatible and 32-bit in the meaning of ia32 applications unless otherwise specified.

Difference between native and compat applications

From the CPU's point of view, 32-bit compatibility mode applications differ to 64-bit application by current CS (code segment selector): if corresponding value of L-bit from flags of entry in descriptors table is set the CPU will be in 64-bit mode when this segment descriptor is being used. There are some other differences between 32 and 64-bit selectors, one can read about them in the article "The 0x33 Segment Selector (Heavens Gate)". Code selectors for both bits are defined in kernel headers as __USER32_CS and __USER_CS and corresponds to descriptors in GDT (Global Descriptors Table). One can change 64-bit mode to compatibility mode by swapping CS value (e.g., with longjump).

From the Linux kernel's point of view, applications differ by values set during exec of application such as mmap_base or thread info flags TIF_ADDR32/TIF_IA32/TIF_X32. Both native and compat applications can do 32 or 64-bit syscalls.

Approaches to C/R compatible applications

C/R of compatible applications can be done differently, this section describes cons/pros of each, to address decision why C/R of 32-bit tasks done that way and not some other.

Restore with exec() of 32-bit dummy binary vs from 64-bit CRIU

Restore of 32-bit application can be done with some daemon that runs in 32-bit mode and communicates with CRIU binary (or 32-bit CRIU subprocess).
Pros:

• no kernel patches expected (not quite true: vDSO mremap() still needed support)

Cons:

• CRIU code base does not have special restore daemon to communicate with - code needs to be reworked
• 64-bit app can have 32-bit child, which could be a parent to 64-bit and so on - need to re-exec native 64-bit CRIU from 32-bit dummy (or 32-bit CRIU)
• need to send to the daemon properties of restoring processes, open fds to images, share memory with parsed ps_tree and so on... The number of IPC calls will slow down restore
• restoring becomes more complicated, and if looking forward to restoring user/pid sub-namespaces, it will be too entangled
• no optimized inheritance for task's properties those erase with exec()

Needs to be done (TODO)