How to submit patches

2020-01-29T09:35:02Z

Xemul: /* Mail patches */

== Set up working environment ==

Although criu could be run as non-root (see [[Security]]), development is better to be done as root. For example, some tests require root. So, it would be a good idea to set up some recent Linux distro on a virtual machine.

== Get the source code ==

The CRIU sources are tracked by Git. Official CRIU repo is at [https://github.com/checkpoint-restore/criu https://github.com/checkpoint-restore/criu].

The repository may contain multiple branches. Development happens in the '''criu-dev''' branch.

To clone CRIU repo and switch to the proper branch, run:

<pre><nowiki>
git clone https://github.com/checkpoint-restore/criu criu
cd criu
git checkout criu-dev
</nowiki></pre>

== Compile ==

First, you need to install compile-time dependencies. Check [[Installation#Dependencies]] for more info.

To compile CRIU, run:

make

This should create the <code>./criu/criu</code> executable.

== Edit the source code ==

If you use ctags, you can generate the ctags file by running

make tags

When you change the source code, please keep in mind the following code conventions:

* we prefer tabs and indentations to be 8 characters width
* consider reading [https://www.kernel.org/doc/Documentation/process/coding-style.rst Linux kernel coding style].

Other conventions can be learned from the source code itself. In short, make sure your new code
looks similar to what is already there.

== Test your changes ==

CRIU comes with an extensive test suite. To check whether your changes introduce any regressions, run

make test

The command runs [[ZDTM Test Suite]]. Check for any error messages produced by it.

In case you'd rather have someone else run the tests, you can use travis-ci for your
own github fork of CRIU. It will check the compilation for various supported platforms,
as well as run most of the tests from the suite. See https://travis-ci.org/checkpoint-restore/criu
for more details.

== Make a patch ==

To create a patch, run

git format-patch --signoff origin/criu-dev

You might need to read GIT documentation on how to prepare patches
for mail submission. Take a look at http://book.git-scm.com/ and/or
http://git-scm.com/documentation for details. It should not be hard
at all.

We recommend to post patches using <code>git send-email</code>

git send-email --cover-letter --no-chain-reply-to --annotate \
--confirm=always --to=criu@openvz.org criu-dev

Note that the <code>git send-email</code> subcommand may not be in
the main git package and using it may require installation of a
separate package, for example the "git-email" package in Fedora and
Debian.

If this is your first time using git send-email, you might need to
configure it to point it to your SMTP server with something like:

git config --global sendemail.smtpServer stmp.example.net

If you get tired of typing <code>--to=criu@openvz.org</code> all the time,
you can configure that to be automatically handled as well:

git config sendemail.to criu@openvz.org

If a developer is sending another version of the patch (e.g. to address
review comments), they are advised to note differences to previous versions
after the <code>---</code> line in the patch so that it helps reviewers but
doesn't become part of git history. Moreover, such patch needs to be prefixed
correctly with <code>--subject-prefix=PATCHv2</code> appended to
<code>git send-email</code> (substitute <code>v2</code> with the correct
version if needed though).

== Sign your work ==

To improve tracking of who did what, we ask you to sign off the patches
that are to be emailed.

The sign-off is a simple line at the end of the explanation for the
patch, which certifies that you wrote it or otherwise have the right to
pass it on as an open-source patch. The rules are pretty simple: if you
can certify the below:

<div class="toccolours mw-collapsible mw-collapsed" style="width: 46em;">
'''Developer's Certificate of Origin 1.1'''
<div class="mw-collapsible-content">
By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
</div></div>
then you just add a line saying

Signed-off-by: Random J Developer <random at developer.example.org>

using your real name (please, no pseudonyms or anonymous contributions if
it possible).

Hint: you can use <code>git commit -s</code> to add Signed-off-by line to your
commit message. To append such line to a commit you already made, use
<code>git commit --amend -s</code>.

<div class="toccolours mw-collapsible mw-collapsed" style="width: 46em;">
'''Example patch message'''
<div class="mw-collapsible-content">

From: Random J Developer <random at developer.example.org>
Subject: [PATCH] Short patch description

Long patch description (could be skipped if patch
is trivial enough)

Signed-off-by: Random J Developer <random at developer.example.org>
---
Patch body here
</div></div>

== Mail patches ==

The patches should be sent to CRIU development mailing list, <code>criu AT openvz.org</code>. Note that you need to be subscribed first in order to post. The list web interface is available at https://openvz.org/mailman/listinfo/criu; you can also use standard mailman aliases to work with it.

Please make sure the email client you're using doesn't screw your patch (line wrapping and so on).

{{Note| When sending a patch set that consists of more than one patch, please, push your changes in your local repo and provide the URL of the branch in the cover-letter}}

== Wait for response ==

Be patient. Most CRIU developers are pretty busy people so if
there is no immediate response on your patch — don't be surprised,
sometimes a patch may fly around a week before it gets reviewed.

== Continuous integration ==

''Main article: [[Continuous integration]]''

CRIU tests are run for each series sent to the mailing list. If you get a message from our patchwork that patches failed to pass the tests, you have to investigate what is wrong.

We also recommend you to [[Continuous integration#Enable Travis CI for your repo|enable Travis CI for your repo]] to check patches in your git branch, before sending them to the mailing list.

[[Category:Development]]

Google Summer of Code 2020

2020-01-29T08:24:46Z

Xemul:

This page lists accepted summary for the GSoC-2020 campaign.

== Links ==
* [https://summerofcode.withgoogle.com Google Summer of Code page]
* [[Google Summer of Code Ideas|List of ideas]].

== Status ==

==Student final submissions==

[[Category:GSoC]]

Main Page

2020-01-29T08:14:13Z

Xemul:

<div style="float: {{{1|right}}}">
{{Download box|left}}
[[Image:Willet_texas_april2003_1_1280.png|right|178px]]
</div>
__NOTOC__
<big>Welcome to CRIU, a project to implement checkpoint/restore functionality for Linux.

Checkpoint/Restore In Userspace, or CRIU (pronounced kree-oo, IPA: /krɪʊ/, Russian: криу), is a Linux software. It can freeze a running container (or an individual application) and checkpoint its state to disk. The data saved can be used to restore the application and run it exactly as it was during the time of the freeze. Using this functionality, application or container live migration, snapshots, remote debugging, and [[usage scenarios|many other things]] are now possible.

CRIU started as a project of Virtuozzo, and grew with the tremendous help from the [[community]]. It is currently used by (integrated into) OpenVZ, [[LXC]]/LXD, [[Docker]], Podman, and [[Integration|other software]], and [[packages|packaged for many Linux distributions]].
</big>
{{Like}}
<br clear="both">

<div class="m_right">
{{News block 2}}
</div>

<div class="m_left">
== Using ==

<big>
;Getting [[packages]] for your distribution
: Or try manual [[installation]] to have CRIU on your system
</big>

;[[CLI]], [[RPC]] and [[C API]]
: Three ways to start using the C/R functionality. [[:Category:API|More info]] about APIs.

;[[Usage scenarios]]
: Ideas how criu can be used (some are crazy indeed)

;[[:Category:HOWTO]]
: Collection of real world examples of how to use CRIU. Some are complex, some are not. HOW TO dump a [[simple loop]] might be the best one to start with. Also a set of [[asciinema]] records for real-life examples.

;[[FAQ]] & [[When C/R fails]]
: A sort of troubleshooting guide

;[[What can change after C/R]]
: CRIU cannot (yet) save and restore every single bit of tasks' state. This page describes what bits visible through standard kernel API are such.

;[[What cannot be checkpointed]]
: What an application could do to make CRIU refuse to dump it.

;[[Contacts]]
: Ways to communicate with CRIU community

</div>

<div class="m_center">
== Developing ==
If you're interested in CRIU development, please subscribe to the criu mailing list: https://lists.openvz.org/mailman/listinfo/criu

;[[Images]]
: Description of image files format

;[[Plugins]]
: CRIU can call plugins provided by people

;[[Upstream kernel commits]]
: Mainline kernel commits tracker

;[[Recent commits]]
: CRIU tool repository commits

;[[Manpages]]
: Kernel's manpages commits tracker

;[[ZDTM Test Suite]]
: Zero downtime test suite

;[[Todo|TODO]]
: Current TODO list

;[[User namespace]]
: Implementing user namespace support

;[[Postulates]]
: What to keep in mind when writing new code

;[https://coveralls.io/github/checkpoint-restore/criu Code coverage results]
: Shows how zdtm run covers the criu code paths

;[[How to submit patches]]
:

</div>

<br clear="both">
<div class="m_left">
== Under the hood ==
* [[Checkpoint/Restore]]
* [[:Category:Under the hood]]
* [[:Category:Network]]
* [[:Category:Files]]
* [[:Category:Memory]]
* [[Pending signals]]
* [[Stages of restoring]]
* [[Code blobs]]
* [[Comparison to other CR projects]]
</div>

<div class="m_center">
== External links ==
{{:Articles}}
</div>

<div class="m_right">
== Misc ==
* [[Podcasts]] and other audio/video interviews
* Project [[history]]
* [[Logo]] description
* [[Events]]
* [[CRIU acronym fun]]
</div>

Xemul: /* Improvements */

[[Image:hummingbird.png|400px|right]]
{{Release|3.13}}

=== New features ===
* VDSO: arm32 support
* Add TLS support for [[page server]] communications
* "Ignore" mode for {{Opt|--manage-cgroups}}
* Restore SO_BROADCAST option for inet [[sockets]]

=== Bugfixes ===
* Auxiliary events were left in inotify queues
* Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
* Memory and resource leakage were detected by coverity, cppcheck and clang

=== Improvements ===
* Use gettimeofday() directly from vdso for restore timings
* Reformat all .py code into pep8 style

2019-03-27T10:23:03Z

Xemul: /* Anonymise image files */

Google Summer of Code (GSoC) is a global program that offers post-secondary students an opportunity to be paid for contributing to an open source project over a three month period.

This page contains project ideas for upcoming Google Summer of Code.

== Suggested ideas ==

=== Post-copy for shared memory and hugetlbfs ===

'''Summary:''' extend post-copy memory restore and migration to support shared memory and hugetlbfs.

CRIU relies on [[Userfaultfd]] mechanism in the Linux kernel to implement the demand paging in userspace and allow post-copy memory (or lazy) [[Lazy_migration|migration]]. However, currently this support is limited to anonymous private memory mappings, although kernel also supports shared memory areas and hugetlbfs backed memory.

The shared memory support for lazy migration can be added to CRIU without kernel modifications, while proper handling of hugetlbfs would require userfaultfd callbacks for [http://man7.org/linux/man-pages/man2/fallocate.2.html fallocate(PUNCH_HOLE)] and [http://man7.org/linux/man-pages/man2/madvise.2.html madvise(MADV_REMOVE)] system calls.

'''Links:'''
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/userfaultfd.html Userfaultfd]
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html hugetlbfs]
'''Details:'''
* Skill level: most probably advanced?
* Language: C
* Mentor: Mike Rapoport <rppt@linux.ibm.com> / Andrey Vagin <avagin@gmail.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>

=== Optimize logging engine ===

'''Summary:''' CRIU puts a lots of logs when doing its job. Logging is done with simple fprintf function. They are typically useless, but ''if'' some operation fails -- the logs are the only way to find what was the reason for failure.

At the same time the printf family of functions is known to take some time to work -- they need to scan the format string for %-s and then convert the arguments into strings. If comparing criu dump with and without logs the time difference is notable (15%-20%), so speeding the logs up will help improve criu performance.

One of the solutions to the problem might be binary logging. The problem with binary logs is the amount of efforts to convert existing logs to binary form. Preferably, the switch to binary logging either keeps existing log() calls intact, either has some automatics to convert them.

The option to keep log() calls intact might be in pre-compilation pass of the sources. In this pass each <code>log(fmt, ...)</code> call gets translated into a call to a binary log function that saves <code>fmt</code> identifier copies all the args ''as is'' into the log file. The binary log decode utility, required in this case, should then find the fmt string by its ID in the log file and print the resulting message.

'''Links:'''
* [[Better logging]]

'''Details:'''
* Skill level: intermediate
* Language: C, though decoder/preprocessor can be in any language
* Mentor: Andrei Vagin <avagin@gmail.com> / Pavel Emelyanov <xemul@openvz.org>
* Suggested by: Andrei Vagin <avagin@gmail.com>

=== Add support for checkpoint/restore of CORK-ed UDP socket ===

'''Summary:''' Support C/R of corked UDP socket

There's UDP_CORK option for sockets. As man page says:
<pre>
If this option is enabled, then all data output on this socket
is accumulated into a single datagram that is transmitted when
the option is disabled. This option should not be used in
code intended to be portable.
</pre>

Currently criu refuses to dump this case, so it's effectively a bug. Supporting
this will need extending the kernel API to allow criu read back the write queue
of the socket (see [[TCP connection|how it's done]] for TCP sockets, for example). Then
the queue is written into the image and is restored into the socket (with the CORK
bit set too).

'''Links:'''
* https://github.com/checkpoint-restore/criu/issues/409
* [[Sockets]], [[TCP connection]]
* [[https://groups.google.com/forum/#!topic/comp.os.linux.networking/Uz8PYiTCZSg UDP cork explained]]

'''Details:'''
* Skill level: intermediate (+linux kernel)
* Language: C
* Mentor: Pavel Emelianov <xemul@virtuozzo.com>
* Suggested by: Pavel Emelianov <xemul@virtuozzo.com>

=== Optimize the pre-dump algorithm ===

'''Summary:''' Optimize the pre-dump algorithm to avoid pinning to many memory in RAM

Current [[CLI/cmd/pre-dump|pre-dump]] mode is used to write task memory contents into image
files w/o stopping the task for too long. It does this by stopping the task, infecting it and
draining all the memory into a set of pipes. Then the task is cured, resumed and the pipes'
contents is written into images (maybe a [[page server]]). Unfortunately, this approach creates
a big stress on the memory subsystem, as keeping all memory in pipes creates a lot of unreclaimable
memory (pages in pipes are not swappable), as well as the number of pipes themselves can be huge, as
one pipe doesn't store more than a fixed amount of data (see pipe(7) man page).

A solution for this problem is to use a sys_read_process_vm() syscall, which will mitigate
all of the above. To do this we need to allocate a temporary buffer in criu, then walk the
target process vm by copying the memory piece-by-piece into it, then flush the data into image
(or page server), and repeat.

Ideally there should be sys_splice_process_vm() syscall in the kernel, that does the same as
the read_process_vm does, but vmsplices the data

'''Links:'''
* [[Memory pre dump]]
* https://github.com/checkpoint-restore/criu/issues/351
* [[Memory dumping and restoring]], [[Memory changes tracking]]
* [http://man7.org/linux/man-pages/man2/process_vm_readv.2.html process_vm_readv(2)] [http://man7.org/linux/man-pages/man2/vmsplice.2.html vmsplice(2)] [https://lkml.org/lkml/2018/1/9/32 RFC for splice_process_vm syscall]

'''Details:'''
* Skill level: advanced
* Language: C
* Mentor: Pavel Emelianov <xemul@virtuozzo.com>
* Suggested by: Pavel Emelianov <xemul@virtuozzo.com>

=== Anonymise image files ===

'''Summary:''' Teach [[CRIT]] to remove sensitive information from images

When reporting a BUG it may be not acceptable for the reporter to send us raw images, as they may contain sensitive data. Need to teach CRIT to "anonymise" images for publication.

List of data to shred:

* Memory contents. For the sake of investigation, all the memory contents can be just removed. Only the sizes of pages*.img files are enough.
* Paths to files. Here we should keep the paths relations to each other. The simplest way seem to be replacing file names with "random" (or sequential) strings, BUT (!) keeping an eye on making this mapping be 1:1. Note, that file paths may also sit in sk-unix.img.
* Registers.
* Process names. (But relations should be kept).
* Contents of streams, i.e. pipe/fifo data, sk-queue, tcp-stream, tty data.
* Ghost files.
* Tarballs with tmpfs-s.
* IP addresses in sk-inet-s, ip tool dumps and net*.img.

'''Links:'''
* [[Anonymize image files]]
* https://github.com/checkpoint-restore/criu/issues/360
* [[CRIT]], [[Images]]
* External links to mailing lists or web sites

'''Details:'''
* Skill level: beginner
* Language: Python
* Mentor: Pavel Emelianov <xemul@virtuozzo.com>
* Suggested by: Pavel Emelianov <xemul@virtuozzo.com>

=== Porting crit functionalities in GO ===

'''Summary:''' Implement image view and manipulation in Go

CRIU's checkpoint images are stored on disk using protobuf. For easier analysis of checkpoint files CRIU has a tool called [[CRIT|CRiu Image Tool (CRIT)]]. It can display/decode CRIU image files from binary protobuf to JSON as well as encode JSON files back to the binary format. With closer integration of CRIU in container runtimes it becomes important to be able to view the CRIU output files. Either for manipulation before restoring or for reading checkpoint statistics (memory pages written to disk, memory pages skipped, process downtime).

Currently CRIT is implemented in Python, for easier integration in other Go projects it is important to have image manipulation and analysis available from GO. This means we need a Go based library to read/modify/write/encode/decode CRIU's image files. Based on this library a Go based implementation of CRIT would be useful.

'''Links:'''
* [[CRIT]]
* Possible use case see LXD: https://github.com/lxc/lxd/blob/cb55b1c5a484a43e0c21c6ae8c4a2e30b4d45be3/lxd/migrate_container.go#L179
* https://github.com/lxc/lxd/pull/4072
* https://github.com/checkpoint-restore/go-criu/blob/master/phaul/stats.go

'''Details:'''
* Skill level: beginner
* Language: Go
* Mentor: Adrian Reber <areber@redhat.com>
* Suggested by: Adrian Reber <areber@redhat.com>

=== Memory changes tracking with userfaultfd-WP ===

'''Summary:''' add ability to track memory changes of the snapshotted processes using userfaultfd-WP

Currently CRIU uses [[Memory_changes_tracking|Soft-dirty]] mechanism in Linux kernel to track memory changes.
This mechanism can be complemented (or even completely replaced) with recently proposed write protection support for
userfaultfd (userfaultfd-WP).

Userfault allows implementation of paging in userspace. It allows an application to receive notifications about page faults and provide the desired memory contents for the faulting pages. In the current upstream kernels only missing page faults are supported, but there is an ongoing work to allow notifications for write faults as well. Using such notifications it would be possible to precisely track memory changes during pre-dump iterations. This approach may prove to be more efficient than soft-dirty.

'''Links:'''
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/userfaultfd.html Userfaultfd]
* [https://github.com/xzpeter/linux/tree/uffd-wp-merged Userfaultfd-WP]
* [https://www.kernel.org/doc/html/latest/admin-guide/mm/soft-dirty.html?highlight=soft%20dirty Soft-Dirty]
* https://lwn.net/Articles/777258/

'''Details:'''
* Skill level: most probably advanced?
* Language: C
* Mentor: Mike Rapoport <rppt@linux.ibm.com>
* Suggested by: Mike Rapoport <rppt@linux.ibm.com>

[[Category:GSoC]]
[[Category:Development]]

Anonymize image files

2019-03-27T10:22:39Z

Xemul: Created page with "When resolving a bug from user criu team often needs the images that caused the problem, since using pure logs is not always helps to understand the cause of the problem. At t..."

When resolving a bug from user criu team often needs the images that caused the problem, since using pure logs is not always helps to understand the cause of the problem. At the same time, sharing images with developers may be impossible since they may contain
sensitive information -- memory contents with passwords/keys/secrets, paths to files on dist that may be not-to-be-shared.

So all the secrets must be removed from images, but still we should keep the possibility to work with them in terms of objects criu uses for restore. I.e. everything that sits in the anonymous images must be open-able, connect-able, mmap-able, etc-able.

== List of data to shred ==

* Memory contents. For the sake of investigation, all the memory contents can be just removed. Only the sizes of pages*.img files and pagemaps are enough.
* Paths to files. Here we should keep the paths relations to each other. The simplest way seem to be replacing file names with "random" (or sequential) strings, BUT (!) keeping an eye on making this mapping be 1:1. Note, that file paths may also sit in sk-unix.img.
* Registers.
* Process names. (But pid-to-pid and sid/pgid relations should be kept).
* Contents of streams, i.e. pipe/fifo data, sk-queue, tcp-stream, tty data.
* Ghost files.
* Tarballs with tmpfs-s.
* IP addresses in sk-inet-s, ip tool dumps and net*.img.

== See also ==
* [[Images]]
* [[CRIT]]

[[Category:Images]]