Changes

3,459 bytes added ,  12:22, 18 April 2022
no edit summary
Line 76: Line 76:        +
== How CRIU handles rseq ==
 +
 +
CRIU handles the rseq differently depending on the particular case. Let's classify and cover all of them.
 +
 +
# the process is not inside the rseq critical section
 +
# the process is inside the rseq CS
 +
## <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
 +
## <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code>
 +
 +
=== the process is not inside the rseq critical section ===
 +
 +
Simplest case. Process just have <code>struct rseq</code> registered in the kernel but currently instruction pointer (IP) not inside CS.
 +
 +
==== Dump ====
 +
We need only to determine where the <code>struct rseq</code> is and dump its address length and signature.
 +
To achieve that we use special ptrace handle <code>PTRACE_GET_RSEQ_CONFIGURATION</code> (refer to the <code>dump_thread_rseq</code> function).
 +
 +
==== Restore ====
 +
We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
 +
 +
=== inside CS: <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code> ===
 +
 +
The process was caught with IP inside CS. Can we act as before? So, dump <code>struct rseq</code> address, restore it, and so on. No, we can't.
 +
The reason is that CRIU saves IP as it was during the dump. But the rseq semantic is to jump to abort handler if CS execution was interrupted.
 +
In this particular case we have <code>flags</code> equal to <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
 +
it means that if CS will be interrupted by the preeption, migration (<code>0</code>) or migration (<code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code>) or preemption (<code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>)
 +
the kernel will fixup IP of the process to the abort handler address.
 +
 +
When we dump the process using CRIU it will just save IP as it was and restore it. That's a serious problem and this may break the user application (even cause crash!).
 +
 +
Lets see <code>fixup_thread_rseq</code> function:
 +
<pre>
 +
if (task_in_rseq(rseq_cs, TI_IP(core))) {
 +
struct pid *tid = &item->threads[i];
 +
 +
...
 +
 +
pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
 +
tid->real);
 +
 +
...
 +
 +
if (!(rseq_cs->flags & RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL)) {
 +
pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
 +
tid->real);
 +
 +
TI_IP(core) = rseq_cs->abort_ip;
 +
 +
if (item->pid->real == tid->real) {
 +
compel_set_leader_ip(dmpi(item)->parasite_ctl, rseq_cs->abort_ip);
 +
} else {
 +
compel_set_thread_ip(dmpi(item)->thread_ctls[i], rseq_cs->abort_ip);
 +
}
 +
}
 +
}
 +
</pre>
 +
 +
It checks that process IP inside CS and fixes it up to the abort handler IP as the kernel does.
 +
 +
==== Dump ====
 +
 +
 +
==== Restore ====
 +
 +
 +
=== inside CS: <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> ===
 +
 +
Rare case, but we support it too.
 +
 +
==== Dump ====
 +
 +
 +
==== Restore ====
 +
 +
 +
== TODO ==
 +
 +
* tests for all architectures (right now we have ZDTM tests only for x86_64)
 +
* improvement support of built-in rseq for non-Glibc libraries
 +
* pre-dump tests (?)
 +
* leave-running tests (?)
 +
* crfail test
 +
* threaded test
    
== Useful links ==
 
== Useful links ==