37
edits
Changes
Jump to navigation
Jump to search
Line 9:
Line 9: − + + + + + + − When criu dumps processes it also dumps files that are opened by them. It does this by saving file names by which the files are accessible. But sometimes files can have no names. It may happen if a task opened a file and then removed it. To dump this file criu cannot save its name (because the name doesn't exist). Instead criu saves the whole file. This is called "ghost file". Since saving the whole file is very expensive (copying lots of data on disk) criu limits the maximum size of a ghost file. The latter is also not good, because there are "sparse" files, that are large in size, but may be small from the real disk usage perspective. The goal of the task is to support sparse ghost files, i.e. limit the size of the ghost not by its length but by disk usage and when copying the data detect the used blocks and save only those.+ − + − + − '''Links:'''+ − *[https://en.wikipedia.org/wiki/Sparse_file Sparse files] − *[[Dumping files]] − *[[Invisible files]] − *[https://www.kernel.org/doc/html/latest/filesystems/fiemap.html Fiemap ioctl] − − * Mentor: Pavel Emelyanov <ovzxemul@gmail.com> − * Mentor: Pavel Tikhomirov <ptikhomirov@virtuozzo.com> − * Suggested by: Pavel Emelyanov <ovzxemul@gmail.com> − − === Optimize logging engine === − − '''Summary:''' CRIU puts a lots of logs when doing its job. Logging is done with simple fprintf function. They are typically useless, but ''if'' some operation fails -- the logs are the only way to find what was the reason for failure. − − At the same time the printf family of functions is known to take some time to work -- they need to scan the format string for %-s and then convert the arguments into strings. If comparing criu dump with and without logs the time difference is notable (15%-20%), so speeding the logs up will help improve criu performance. − − One of the solutions to the problem might be binary logging. The problem with binary logs is the amount of efforts to convert existing logs to binary form. Preferably, the switch to binary logging either keeps existing log() calls intact, either has some automatics to convert them. − − The option to keep log() calls intact might be in pre-compilation pass of the sources. In this pass each <code>log(fmt, ...)</code> call gets translated into a call to a binary log function that saves <code>fmt</code> identifier copies all the args ''as is'' into the log file. The binary log decode utility, required in this case, should then find the fmt string by its ID in the log file and print the resulting message. − − '''Links:''' − * [[Better logging]] − − '''Details:''' − * Skill level: intermediate − * Language: C, though decoder/preprocessor can be in any language − * Expected size: 350 hours − * Mentor: Pavel Emelyanov <ovzxemul@gmail.com> + Line 70:
Line 49: + Line 77:
Line 57: − + − * Suggested by: Pavel Emelianov <ovzxemul@gmail.com> Line 101:
Line 80: − * Suggested by: Alexander Mikhalitsyn <alexander@mihalicyn.com> − − === Add support for memfd_secret file descriptors === − − '''Summary:''' Support C/R of memfd_secret descriptors − − There is memfd_secret syscall which allows user to open − special memfd which is backed by special memory range which − is inaccessible by another processes (and the kernel too!). − − At the moment CRIU can't dump processes that have memfd_secret's opened. − − '''Links:''' − * https://lwn.net/Articles/865256/ − − '''Details:''' − * Skill level: intermediate − * Language: C − * Expected size: 350 hours − * Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>, Mike Rapoport <mike.rapoport@gmail.com> Line 138:
Line 97: − + − − === CGroup-v2 support === − − '''Summary:''' cgroup is a mechanism to organize processes hierarchically and distribute system resources along the hierarchy in a controlled and configurable manner. cgroup v2 is a new version of the cgroup file system. Unlike v1, cgroup v2 has only single hierarchy. CRIU has to dump/restore a container cgroup hierarchy along with all per-cgroup options. The cgroupv2 support in CRIU has to be compatible with Docker, containerd and cri-o. − − '''Links:''' − * [[CGroups]] − * https://github.com/checkpoint-restore/criu/issues/252 − * https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html − '''Details:''' − * Skill level: intermediate − * Language: C − * Expected size: 350 hours − * Mentor: Andrei Vagin <avagin@gmail.com> − * Suggested by: Andrei Vagin <avagin@gmail.com> − − === Dump shmem in user-mode (unprivileged-mode) === − − CRIU uses /proc/pid/map_files to dump and restore anonymous shared memory regions, but map_files is restricted to the global CAP_SYS_ADMIN capability. In most cases, it is possible to dump/restore shared memory region without map_files and we need to implement this in CRIU. − − '''Links:''' − * [[User-mode]] − − '''Details:''' − * Skill level: intermediate − * Language: C − * Expected size: 350 hours − * Suggested by: Andrei Vagin <avagin@gmail.com> − * Suggested by: Pavel Emelyanov <ovzxemul@gmail.com> − * Mentor: Pavel Emelyanov <ovzxemul@gmail.com> Line 208:
Line 137: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Line 225:
Line 222: − * Suggested by: Pavel Emelyanov <ovzxemul@gmail.com> − * Mentor: Pavel Emelyanov <ovzxemul@gmail.com> Line 243:
Line 238: − + − + Line 273:
Line 268: − * Mentor: Pavel Emelianov <xemul@virtuozzo.com>+ − * Suggested by: Pavel Emelianov <xemul@virtuozzo.com>
→Add support for checkpoint/restore of CORK-ed UDP socket
== Project ideas ==
== Project ideas ==
=== Support sparse ghosts ===
=== Add support for memory compression ===
'''Summary:''' Support compression for page images
We would like to support memory page files compression
in CRIU using one of the fastest algorithms (it's matter
of discussion which one to choose!).
This task does not require any Linux kernel modifications
and scope is limited to CRIU itself. At the same time it's
complex enough as we need to touch memory dump/restore codepath
in CRIU and also handle many corner cases like page-server and stuff.
'''Details:'''
'''Details:'''
* Skill level: intermediate
* Skill level: intermediate
* Language: C
* Language: C
* Expected size: 350 hours
* Expected size: 350 hours
* Suggested by: Andrei Vagin <avagin@gmail.com>
* Suggested by: Andrei Vagin <avagin@gmail.com>
* Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>, Andrei Vagin <avagin@gmail.com>
=== Add support for checkpoint/restore of CORK-ed UDP socket ===
=== Add support for checkpoint/restore of CORK-ed UDP socket ===
'''Links:'''
'''Links:'''
* https://github.com/checkpoint-restore/criu/issues/409
* https://github.com/checkpoint-restore/criu/issues/409
* https://github.com/criupatchwork/criu/commit/a532312
* [[Sockets]], [[TCP connection]]
* [[Sockets]], [[TCP connection]]
* [[https://groups.google.com/forum/#!topic/comp.os.linux.networking/Uz8PYiTCZSg UDP cork explained]]
* [[https://groups.google.com/forum/#!topic/comp.os.linux.networking/Uz8PYiTCZSg UDP cork explained]]
* Language: C
* Language: C
* Expected size: 350 hours
* Expected size: 350 hours
* Mentor: Pavel Emelianov <ovzxemul@gmail.com>
* Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>, Pavel Tikhomirov <ptikhomirov@virtuozzo.com>, Andrei Vagin <avagin@gmail.com>
=== Add support for pidfd file descriptors ===
=== Add support for pidfd file descriptors ===
* Expected size: 350 hours
* Expected size: 350 hours
* Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>, Christian Brauner <christian@brauner.io>
* Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>, Christian Brauner <christian@brauner.io>
* Suggested by: Alexander Mikhalitsyn <alexander@mihalicyn.com>
* Suggested by: Alexander Mikhalitsyn <alexander@mihalicyn.com>
* Language: C
* Language: C
* Expected size: 350 hours
* Expected size: 350 hours
* Mentor: Radostin Stoyanov <rstoyanov@fedoraproject.org>
* Mentors: Radostin Stoyanov <rstoyanov@fedoraproject.org>, Prajwal S N <prajwalnadig21@gmail.com>
* Suggested by: Adrian Reber <areber@redhat.com>
* Suggested by: Adrian Reber <areber@redhat.com>
=== Files on detached mounts ===
=== Files on detached mounts ===
* Skill level: intermediate
* Skill level: intermediate
* Language: C
* Language: C
* Expected size: 350 hours
* Mentor: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
* Mentor: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
* Suggested by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
* Suggested by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
=== Checkpointing of POSIX message queues ===
'''Summary:''' Add support for checkpoint/restore of POSIX message queues
POSIX message queues are a widely used inter-process communication mechanism. Message queues are implemented as files on a virtual filesystem (mqueue), where a file descriptor (message queue descriptor) is used to perform operations such as sending or receiving messages. To support checkpoint/restore of POSIX message queues, we need a kernel interface (similar to [https://github.com/checkpoint-restore/criu/commit/8ce9e947051e43430eb2ff06b96dddeba467b4fd MSG_PEEK]) that would enable the retrieval of messages from a queue without removing them. This project aims to implement such an interface that allows retrieving all messages and their priorities from a POSIX message queue.
'''Links:'''
* https://github.com/checkpoint-restore/criu/issues/2285
* https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/ipc/mqueue.c
* https://www.man7.org/tlpi/download/TLPI-52-POSIX_Message_Queues.pdf
'''Details:'''
* Skill level: intermediate
* Language: C
* Expected size: 350 hours
* Mentors: Radostin Stoyanov <rstoyanov@fedoraproject.org>, Pavel Tikhomirov <ptikhomirov@virtuozzo.com>, Prajwal S N <prajwalnadig21@gmail.com>
* Suggested by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
=== Kubernetes operator for managing container checkpoints ===
'''Summary:''' Develop a Kubernetes operator that automates the management of container checkpoints
Container checkpointing has recently been introduced as an alpha feature in Kubernetes.
To enable this feature, the kubelet API was extended with an endpoint that enables the
creation of checkpoints for individual containers. By default, all container checkpoints
are stored as tar archives in <code>/var/lib/kubelet/checkpoints</code> using the following
file name format: <code>checkpoint-<pod-name>_<namespace-name>-<container-name>-<timestamp>.tar</code>.
However, the current implementation does not provide a mechanism for limiting the number
of checkpoints, which may lead to filling up all existing disk space. This project aims to
develop a Kubernetes operator that automates the management of checkpoints and provides
a garbage collection mechanism to discard obsolete checkpoints.
'''Links:'''
* https://github.com/checkpoint-restore/checkpoint-restore-operator
* https://kubernetes.io/docs/reference/node/kubelet-checkpoint-api/
* https://kubernetes.io/blog/2022/12/05/forensic-container-checkpointing-alpha/
* https://kubernetes.io/blog/2023/03/10/forensic-container-analysis/
* https://github.com/kubernetes/kubernetes/pull/115888
* https://github.com/kubernetes/enhancements/issues/2008
'''Details:'''
* Skill level: intermediate
* Language: Go
* Expected size: 350 hours
* Mentors: Adrian Reber <areber@redhat.com>, Radostin Stoyanov <rstoyanov@fedoraproject.org>, Prajwal S N <prajwalnadig21@gmail.com>
* Suggested by: Adrian Reber
== Suspended project ideas ==
== Suspended project ideas ==
Listed here are tasks that seem suitable for GSoC, but currently do not have anybody to mentor it.
Listed here are tasks that seem suitable for GSoC, but currently do not have anybody to mentor it.
=== Optimize logging engine ===
'''Summary:''' CRIU puts a lots of logs when doing its job. Logging is done with simple fprintf function. They are typically useless, but ''if'' some operation fails -- the logs are the only way to find what was the reason for failure.
At the same time the printf family of functions is known to take some time to work -- they need to scan the format string for %-s and then convert the arguments into strings. If comparing criu dump with and without logs the time difference is notable (15%-20%), so speeding the logs up will help improve criu performance.
One of the solutions to the problem might be binary logging. The problem with binary logs is the amount of efforts to convert existing logs to binary form. Preferably, the switch to binary logging either keeps existing log() calls intact, either has some automatics to convert them.
The option to keep log() calls intact might be in pre-compilation pass of the sources. In this pass each <code>log(fmt, ...)</code> call gets translated into a call to a binary log function that saves <code>fmt</code> identifier copies all the args ''as is'' into the log file. The binary log decode utility, required in this case, should then find the fmt string by its ID in the log file and print the resulting message.
'''Links:'''
* [[Better logging]]
'''Details:'''
* Skill level: intermediate
* Language: C, though decoder/preprocessor can be in any language
* Expected size: 350 hours
* Suggested by: Andrei Vagin
* Mentors: Alexander Mikhalitsyn <alexander@mihalicyn.com>
=== IOUring support ===
=== IOUring support ===
* Skill level: expert (+linux kernel)
* Skill level: expert (+linux kernel)
* Expected size: 350 hours
* Expected size: 350 hours
=== Add support for SPFS ===
=== Add support for SPFS ===
* Skill level: expert
* Skill level: expert
* Language: C
* Language: C
* Mentor: Alexander Mikhalitsyn <alexander@mihalicyn.com> / <alexander.mikhalitsyn@virtuozzo.com>
* Mentor: Alexander Mikhalitsyn <alexander@mihalicyn.com>
* Suggested by: Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>
* Suggested by: Alexander Mikhalitsyn <alexander@mihalicyn.com>
* Skill level: beginner
* Skill level: beginner
* Language: Python
* Language: Python
[[Category:GSoC]]
[[Category:GSoC]]
[[Category:Development]]
[[Category:Development]]