31
edits
Changes
no edit summary
== How CRIU handles rseq ==
CRIU handles the rseq differently depending on the particular case. Let's classify and cover all of them.
# the process is not inside the rseq critical section
# the process is inside the rseq CS
## <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
## <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code>
=== the process is not inside the rseq critical section ===
Simplest case. Process just have <code>struct rseq</code> registered in the kernel but currently instruction pointer (IP) not inside CS.
==== Dump ====
We need only to determine where the <code>struct rseq</code> is and dump its address length and signature.
To achieve that we use special ptrace handle <code>PTRACE_GET_RSEQ_CONFIGURATION</code> (refer to the <code>dump_thread_rseq</code> function).
==== Restore ====
We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
=== inside CS: <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code> ===
The process was caught with IP inside CS. Can we act as before? So, dump <code>struct rseq</code> address, restore it, and so on. No, we can't.
The reason is that CRIU saves IP as it was during the dump. But the rseq semantic is to jump to abort handler if CS execution was interrupted.
In this particular case we have <code>flags</code> equal to <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
it means that if CS will be interrupted by the preeption, migration (<code>0</code>) or migration (<code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code>) or preemption (<code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>)
the kernel will fixup IP of the process to the abort handler address.
When we dump the process using CRIU it will just save IP as it was and restore it. That's a serious problem and this may break the user application (even cause crash!).
Lets see <code>fixup_thread_rseq</code> function:
<pre>
if (task_in_rseq(rseq_cs, TI_IP(core))) {
struct pid *tid = &item->threads[i];
...
pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
tid->real);
...
if (!(rseq_cs->flags & RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL)) {
pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
tid->real);
TI_IP(core) = rseq_cs->abort_ip;
if (item->pid->real == tid->real) {
compel_set_leader_ip(dmpi(item)->parasite_ctl, rseq_cs->abort_ip);
} else {
compel_set_thread_ip(dmpi(item)->thread_ctls[i], rseq_cs->abort_ip);
}
}
}
</pre>
It checks that process IP inside CS and fixes it up to the abort handler IP as the kernel does.
==== Dump ====
==== Restore ====
=== inside CS: <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> ===
Rare case, but we support it too.
==== Dump ====
==== Restore ====
== TODO ==
* tests for all architectures (right now we have ZDTM tests only for x86_64)
* improvement support of built-in rseq for non-Glibc libraries
* pre-dump tests (?)
* leave-running tests (?)
* crfail test
* threaded test
== Useful links ==
== Useful links ==