36
edits
Changes
Jump to navigation
Jump to search
no edit summary
Line 76:
Line 76:
+
== How CRIU handles rseq ==
+
+CRIU handles the rseq differently depending on the particular case. Let's classify and cover all of them.
+
+# the process is not inside the rseq critical section
+# the process is inside the rseq CS
+## <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
+## <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code>
+
+=== the process is not inside the rseq critical section ===
+
+Simplest case. Process just have <code>struct rseq</code> registered in the kernel but currently instruction pointer (IP) not inside CS.
+
+==== Dump ====
+We need only to determine where the <code>struct rseq</code> is and dump its address length and signature.
+To achieve that we use special ptrace handle <code>PTRACE_GET_RSEQ_CONFIGURATION</code> (refer to the <code>dump_thread_rseq</code> function).
+
+==== Restore ====
+We need to take data about the <code>struct rseq</code> from the image (see images/rseq.proto) and register it from the parasite context using the <code>rseq</code> syscall (take a look on <code>restore_rseq</code> in criu/pie/restorer.c)
+
+=== inside CS: <code>flags</code> is <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code> ===
+
+The process was caught with IP inside CS. Can we act as before? So, dump <code>struct rseq</code> address, restore it, and so on. No, we can't.
+The reason is that CRIU saves IP as it was during the dump. But the rseq semantic is to jump to abort handler if CS execution was interrupted.
+In this particular case we have <code>flags</code> equal to <code>0</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code> or <code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>
+it means that if CS will be interrupted by the preeption, migration (<code>0</code>) or migration (<code>RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT</code>) or preemption (<code>RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE</code>)
+the kernel will fixup IP of the process to the abort handler address.
+
+When we dump the process using CRIU it will just save IP as it was and restore it. That's a serious problem and this may break the user application (even cause crash!).
+
+Lets see <code>fixup_thread_rseq</code> function:
+<pre>
+ if (task_in_rseq(rseq_cs, TI_IP(core))) {
+ struct pid *tid = &item->threads[i];
+
+...
+
+ pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
+ tid->real);
+
+...
+
+ if (!(rseq_cs->flags & RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL)) {
+ pr_warn("The %d task is in rseq critical section. IP will be set to rseq abort handler addr\n",
+ tid->real);
+
+ TI_IP(core) = rseq_cs->abort_ip;
+
+ if (item->pid->real == tid->real) {
+ compel_set_leader_ip(dmpi(item)->parasite_ctl, rseq_cs->abort_ip);
+ } else {
+ compel_set_thread_ip(dmpi(item)->thread_ctls[i], rseq_cs->abort_ip);
+ }
+ }
+ }
+</pre>
+
+It checks that process IP inside CS and fixes it up to the abort handler IP as the kernel does.
+
+==== Dump ====
+
+
+==== Restore ====
+
+
+=== inside CS: <code>flags</code> is <code>RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL</code> ===
+
+Rare case, but we support it too.
+
+==== Dump ====
+
+
+==== Restore ====
+
+
+== TODO ==
+
+* tests for all architectures (right now we have ZDTM tests only for x86_64)
+* improvement support of built-in rseq for non-Glibc libraries
+* pre-dump tests (?)
+* leave-running tests (?)
+* crfail test
+* threaded test
== Useful links ==
== Useful links ==