This page contains all the changelogs for all the released CRIU versions. This is mostly useful for doing a search.
v. 3.14
New features
Bugfixes
- Fix C/R ia32 processes on AMD #398
- Fix cross-compilation
- Many fixes here and there
Improvements
- Use clone3() with set_tid to restore processes
- Clean up compel headers.
- Use the new mount API
v. 3.13
New features
Bugfixes
- Auxiliary events were left in inotify queues
- Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
- Memory and resource leakage fixes detected by coverity, cppcheck and clang
Improvements
- Use gettimeofday() directly from vdso for restore timings
- Reformat all .py code into pep8 style
v. 3.12
New features
- build CRIU with Android NDK
- C/R of
- IP RAW sockets
- lsm: dump and restore any SELinux process label
- support restoring ghost files on readonly mounts
Bugfixes
- Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriprots to third-party tools
- small fixes here and there
Improvements
- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
- Changes in the API and ABI (SONAME increased from 1 to 2)
v. 3.11
New features
- C/R of
- epoll: Add support for duped targets
- tun: Add support for multiple net ns
- x86: Support extendable fpu frames
Bugfixes
- mount: Better handling of mount points propagation
- nmk: Make collect-deps to be more precise about targets
- lazy-pages: Don't mark current stack page as lazy
- x86: CPU -- Rework feature testing
- files: Fix O(n^2) restore in terms of the number of fds
- fdstore: Unlimit fdstore queue on start
- mount: Fix regression where
open_mountpoint failed on readonly fs
- page server: Handle partial splicing
- ... lots of small fixes here and there
Improvements
v. 3.10
New features
- Support Python3 in ZDTM and CRIT
- with Python2 ZDTM and CRIT now require
python2-ipaddress to be installed (used to be python2-ipaddr)
- Keep names for UNIX sockets, that are unlinked from the FS
- IPVv6 support for page server
- Set page server socket fd via CLI
- Large pages support for aarch64/ppc64
- C/R of
- Per-thread seccomp chains
Bugfixes
- Failed non-container restore could kill random task on the host
- Failure to dump namespaces was erroneously ignored
- CRIT didn't show cpuinfo image file
- Tasks that got PID-reuse couldn't be dumped iteratively because previous images were missing
v. 3.9
New features
Improvements
Bugfixes
- Random memory corruptions during lazy restore
- Workaround the iptables issue (#469)
- Don't use standard descriptors when tar is running to dump tmpfs mounts
- Fail dump if dump_one_file() fails
- Fill kerndat with zero-s before reading it from cache
- A lot of small fixes here and there
v. 3.8.1
Bugfixes
- FDstore was initialized twice (re-initialized) causing e.g.
--shell-job restore to fail (#460)
v. 3.8
New features
- C/R of
- Multiple network namespaces
- Overmounted tmpfs mounts
- Unix sockets and epoll descriptors in SCM messages
Improvements
Bugfixes
- FP state wasn't reported on Skylake due to a kernel bug
- When compiled with gcc-8 a lot of warnings popped up
- Resource leaked on error paths
- Attributes of sit devices with value 0 were not saved into images (and were restored into default values)
- Tasks with pgid of a zombie hung the resture
- Ghost files on RO bind-mounts of an RW mount couldn't be restored
- Random memory corruptions during lazy restore
v. 3.7
New features
Improvements
- Show criu and kernel versions in logs
- CRIT decodes socket families, protocols and types
- Much less pipes is needed for pre-dump, which is especially useful for big mem migration
Bugfixes
- Files in
/proc/pid/map_file could be opened by non-exiting name (with 0x prefix) and it was fixed in kernel
- CRIU log levels were used to configure logging for libsoccr thus breaking its logs
- Overflow in various IDs caused bad image names
- Compat (32bit) syscalls lost signedness in compel
- Corked sockets lost cork flag
- Preadv() syscall was declared with error which resulted in dump errors on 32-bit processors
- Musl compilation failed
- Ghost files in / dump failed
- Crash when releasing context for ghost files, due to free()-ing shmalloc()-ed area
- Lazy restore could receive partial page and crashed
- Erroneous closing of lazy pages connection caused restore to hang
- Lazy memory fetch restore could start before tasks are restored
v. 3.6
New features
- C/R for
- Files (except for unix sockets, ttys and epolls) sent over unix sockets
- Threads with different creds
- Ipv6 over ipv4 tunnel (SIT device)
Improvements
- CI tests are now also run on Alpine and Fedora Rawhide
Bugfixes
- Some s390x registers were not restored by native sigrestore way
- Overflow when parsing autofs info from /proc file
- Dumps of anon shared memory with sysvipc one raced with each other clashing and corrupting image file names
- The "dumpable" flag was not restored on shmem regions
- Trash bits leaked into image when dumping fsnotify on some kernels
- Lock/unlock of iptables from different criu processes raced with each other
- Closed TCP connection with non-empty send queue blocked the dump
- When
--empty-ns for netns was set on dump only, restore failed (when used from Docker, issue #393)
v. 3.5
New features
- Lazy pages beta support (kernel still needs to be fixed)
Improvements
- More verbose libsoccr
- VDSO rework
- layout is saved in kerndat cache
- PFN detection is relaxed
- restore is faster if kernel provides necessary prctl
Bugfixes
- Task-size calculation could be wrong on s390
- CRIT explore was broken since images format tune-up
- VDSO page could be mis-detected
v. 3.4
New features
- Support for s390x architecture
Improvements
- Unexpected death of restored tasks is reported with more details in logs
- Merged many images containing info about files into one big
files.img
- When helper utility fails (ip, iptables, tar) its name is printed in logs
Bugfixes
- Compilation failed on newer glibcs (ucontext_t)
- Dying helper task could deadlock the restore process
- Install-related makefile variables weren't configurable for distro build
- SIT (ipv6-to-v4 tunnel) presence on host blocked dump of any containers
- Potential NULL dereference when dumping net namespace
- Dump via page server might not work across different criu versions
- Failure to restore a subtask could be ignored by the restore command
- EOF on page-server socket wasn't handled
v. 3.3
New features
Bugfixes
- CRIU didn't compile with glibcs w/o the definition of SIGUNUSED
- Restoring files could race on 64bits
- Restoring a mount in user-namespace could result in broken flags
- C/R on nodes with unified cgroup hierarchy hanged (#252)
- Dumping a character device could crash (detected by ASAN)
- Dumping of tasks with shmem implemented as tmpfs file could fail if the shmem was huge (#230)
v. 3.2.1
Bugfixes
- Restoring a stack fails on recent kernels due to kernel changes #322
- Restoring on a host with LSM profiles failed #323
v. 3.2
Optimizations/improvements
- Invisible files restore is de-serialized
- VMAs restore performance is improved significantly
- Mappings of the same file re-use the descriptor, not re-open it every time
- Not-COW-ed mappings are restored in-place and are not mremmap()ed
- Empty RO mappings are mmap()ed as such and thus not re-mprotect()ed at the end
- More verbosity in case TCP locking fails (#292)
- More verbosity in case VDSO magic mismatch
- Restore or legacy epoll target descriptors and fsnotify marks is unified with common case
Bugfixes
- Restoring fanotify marks from old images (<1.3) dropped the mark
- Binfmt_misc mount could be mounted into wrong place
- Compilation failed with gcc 6.3.0 (#315)
- Waiting helpers could race with sigchild handler and would result in restore failure
- Missing VVAR page in 32bit tasks wasn't skipped and resulted in restore failure
- After restore consumption of files is increased (was fixed as side effect of mmap optimization)
Deprecation/Removal
- Deprecated separate images for fsnotify marks
v. 3.1
New features
- Each boolean option now has the
--no-$option pair
- RSS explorer in CRIT
- Multiple plugins in compel
- Run-time check of 32-mmap BUG on x86
- C/R of
- 32-bit futex robust list on x86
Optimizations/improvements
- Start time is improved significantly with kerndat cache
- Sigaction image is merged into Core
- Unneeded stages are skipped during restore
- Restore w/o namespaces uses host /proc
- Restore w/o namespaces doesn't parse host mounts (not needed)
- Single-threaded tasks do not parse /proc/pid/task/ in vain
- BFD engine is used for more /proc files
- More verbosity in libsoccr
- Fsnotify dump w/o namespaces doesn't walk mounts tree
Bugfixes
- Python bindings left zombie in self-dump mode
- The
last_pid sys-control was reset by restore
- Threads caps were compared with mistake
make install put crit/pycriu to wrong place if DESTDIR was not set (#309)- Fsnotifies C/R w/o namespaces restored with errors
- Inherited control terminal restore was failed (but dump succeeded)
v. 3.0
New features
Optimizations/improvements
- SysVIPC shmem segments are now dumped as any other shmem (taking holes into account and sitting in common memory dumps)
- CRIT show
- decodes socket's states and types and task's states into strings
- prints unix sockets names in more human-readable form
Bugfixes
- Unix sockets' names appeared in logs with mistakes
- Contents of SysVIPC shmem segments was dumped twice
- Dumping of any memory segment more than 4Gigs failed
- Migration of unaligned SysvSHM segment on Armv7 failed
Deprecation/removal
v. 2.12.1
Fixes
- Content for external bind mounts was erroneously dumped, which could lead to dump failures or huuuge images
- Unneeded collection of host mounts on restore could cause restore to fail
v. 2.12
New features
- C/R of
- external TTYs (for Docker C/R)
Optimizations/improvements
- Sanitized the way the
-v works
Fixes
- Checking features via RPC crashed
- Resting pipes in user-namespaces could fail on modern kernels
- Shutdown state for UNIX sockets could be lost on restore
- Dump of huge (over 2Gb) SysV shmem segments didn't work
v. 2.11.1
Fixes
- Page server start via RPC was broken
- Fedora build didn't work
- Ppc64LE restorer switch crashed
v. 2.11
New features
Optimizations/improvements
- More strict checks for extra CLI options
- Report errors when probing locks
- Restorer logs now contain timestamps
Fixes
- Regression: v2.10 was broken on ARM
- Use-after-free when restoring ghost directory
- Array out-of-bound access when restoring VETH device
- Page server exit code could be screwed up
- Clang over-optimized string.h routines resulting in random crashes
- Parasite failed to send FDs via socket on Alpine Linux
- Restore of huge file tables could get stuck
- Restore of epoll in epoll could fail
- Errno value could be lost when reporting failure to restore invisible files
- Dump of sched params didn't work on Alpine
- Restore of huge memory dumps (over 2G) failed
- Installation guessed /lib vs /lib64 with errors
- Migration between xsave and noxsave didn't work for wrong cpu feature being checked
v. 2.10
New features
- C/R of SOCK_PACKET sockets
- Libsoccr -- library for C/R of TCP sockets
Optimizations/improvements
- Logs cleaned up (removed bunch of useless, fixed '\n' in perrors)
- Action scripts errors are printed in logs
- Removed several iovec-s copying over the pagemap code
- Restore degraded linearly on Xen guests. Breakpoints disabled until solution
Fixes
- Py bindings fault on restore error delivery
- Fd leaked on file restore error path
- Fd leaked when restoring invisible files (gets closed with criu exit though)
- Link remap restore could fail on kernels 4.8 and higher
- Impossible to restore after restore error with link remap file in images
- When going daemon a descriptor could be leaked
- Custom setting of
mmap_min_addr could make restore to fail
- Sending pages over UNIX socket could race and fail with EAGAIN
- Error getting ID of /proc/pid/ns/foo link not propagated and could result in bogus NS ID generated
v. 2.9
New features
- CRIU can now be built with clang on all supported architectures
- Ignore missing sysctls on restore with
--weak-sysctl
- C/R overmounted mountpoints
Optimizations/improvements
- Batch restore of memory contents from pages.img files
- Link-remap type for invisible files is explicit in images
- Man page for CRIT
Fixes
- C/R with
--empty-ns still handled iptables configuration
- SCM messages inside UNIX socket got lost after C/R (now dump aborted)
- Empty unixsk.img file appeared when dumping tasks without unix sockets
- Install procedure wasn't PEP-394 compliant
- CRIU blocking netfilter rules were added at the tail of the chain resulting in unlocked TCP connections
- Dump/Restore spurious failures when open() returned 0 descriptor
- When dumping shmem lots of zero pages were written into image files
- Ghost directory with more than zero ghost parents caused restore to fail
- Shared mount could escape to different group on restore
v. 2.8
New features
Optimizations/improvements
- x86 can now be built with clang
- When dumping files useless garbage was sent with descriptors from parasite
- The clear_tid_address and regs are printed in hex with CRIT
- Big code rework for compel (part 1)
- Removed duplicate error messages from opening /proc files
Fixes
- Restoring cgroup NS could use old path prefix
criu check crashed on btrfs mounts- RO external mounts in userns couldn't be restored
- Unmounted on host binfmt_misc could cause dump to fail
- Off-by-one could cause criu crash when dumping shared / bind-mount
- Mount namespace' roots could have flags changed on restore
- Dying tasks could erroneously be tried to dump
- Swapped shared memory pages were not dumped
- Errno value can be sometimes spoiled by RPC
- Restore of netns with newer iproute2 tool could fail
Deprecated
v. 2.7
New features
- Option
--cgroup-root now makes sense on dump too
- CLOCK_BOOTTIME timer supported
Optimizations/improvements
- Output of iptables command leaked into logs for no use
- Helper dev environment installation script for Debian
- Man-page updated and prettified :)
Fixes
- Unmounted binfmt_misc with rules wasn't dumped at all
- Malloc() error could result in crash
- Device cgroup restore could fail restoring empty record
- Some entries in device cgroups were restored twice
- Potential crash when dumping cgroup bindmounts
- Sign error caused dump to fail on btrfs partitions
- Shared mounts with the same mount path failed the dump
- Threads were restored with unshared FS (cwd and root)
- Shared memory changes tracking disabled (regression found)
- Restore of autofs can hang
- LSM profile propagation could be lost
- Mountpoint with lots of options blocked the dump (too small buffer for parsing)
- External slave mount (with external master) blocked the dump
- Mounts with STRICTATIME restored with others flags dropped
Deprecated
- No reg-file entry for TTYs
v. 2.6
New features
Optimizations/improvements
- Use service FD for transport sockets on restore
- Ability to turn pagemap-cache off (some kernels are buggy)
- The
criu --help text has become better
Fixes
- R/O-mounted root could block the dump
- Restore of cgroup.mm.oom_control could fail
- Cgroup fs bind mounts were detected with error
- Unaligned futex-es in parasite could cause dump to crash
- When compiled with gcc-4.9 parasite code crashed
- Failure to freeze cgroup didn't result in aborting of dump
- Wrong ns list was parsed when dumping userns (invisible since nesting works only for mntns)
- Non-inheritable non-tty as stdin caused shell-job restore to erroneously fail
- Error path in
criu dedup could crash
Deprecated
- Per-pid rlimit, itimers and posix-timers
- Separate image for epoll tfds (target file descriptors)
v. 2.5
New features
- C/R
- fs.mqueue.msg*_default sysctls
- Unix sockets with overwritten paths
- Link-remap files in removed directories
Optimizations/improvements
- Micro-optimization on namespace ID evaluation
- Restoring shared files uses one socket instead of per-fd ones
- More verbosity when refusing to dump a file descriptor
Fixes
- Restore could fail on openat() with ENXIO when multiple mnt namespaces get restored
- The criu exec action got broken
- Link-remap and ghost files remained on FS after restore failure
- TCP window could remain clamped after restore resulting in connection lockup/slowdown
- Dump could stuck when injecting a parasite
- The
--timeout option wasn't taken into account when freezing tasks using freezecg
- Race in freezeing/seizing could result in lost tasks
- Memory leaks here and there on error paths
- Double free in xvstrcat (crash)
- VDSO length was mis-calculated
- Symlink on
--root path could make restore erroneously fail
- Potential memory corruption on reading mntns images
- When restoring on systems with low pid_max limit restore could fail
- RO-protected SysV shmem segments could be restored with PROT_EXEC
- File mode of mapped file was evaluated with errors
- Restore of cgroups' mem.swappines and ..use_hierarchy blocked sub-groups creation
- Impossible to restore cgoup mem.swappines default value
- Zombies living in orphan sessions/groups failed the restore
v. 2.4
New features
- Generate core from images
- Ability to forcibly drop half-open TCP connections on C/R
- Ability to specify cgroup ctls to dump via API
- Opened/mapped files' mode is compared between dump and restore times
- C/R of
- AutoFS mountpoints
- New cgroups (perf_event, net_cls, net_prio and pids)
- Memcgroup optional properties
- Devices cgroup
Optimizations/improvements
- Pagemap image entries are cached in memory
Fixes
- Configured kmem cgroup limit restore failed
- Mem cgroup oom_control
- Cgroup's pids.max was not C/R-ed
- Failure to write cgroup property was ignored
- No init PID in pre-dump action script
- Sigactions inheritance didn't work on ARM
- Opened "/proc" dir blocked the dump
- Working with iptables was racy
- Sibling mounts detection error on dump
- Devconf accept_redirects devconf could be restored with errors
- "All" devconfs could be overridden by "Default"
- Name-less unix sockets got auto-bound
- Mode was lost for PTY device file on restore
- Newer protobuf compilers didn't recognize PB files
- External mounts could be remounted with MS_PRIVATE
- Build fail on Alpine Linux
Deprecated/removed
- Per-pid file locks images
- Per-pid fdinfo images
- Ancient pagemap/pages images
v. 2.3
New features
- Ability not to show payload for some objects in CRIT
- Pidfile is written at the end of restore
- Ability to join existing namespaces on restore
- C/R of
- Data sitting in TTYs
- Partially write-protected SysVIPC segments
- Debugfs and tracefs mounts
- Overmounted tmpfs
- IPv6 devconf sysctls
- External block devices
- Unix sockets with mismatched shutdown state
Optimizations/improvements
- Relaxed calculation of AIO ring size
- Tree-based search of tasks by real pid
- Less mem-to-mem copies on restore
- Saner devconf image format
- More verbose explanation of why task cannot be seized
- PID is printed in PIE logs
Fixes
- Too many mmap-ed files blocked the dump
- Potential memory corruption when working with IPv6 sockets
- Overmounted bind mounts could cause restore to fail
- Overmounted bind mounts could result in badly restored mount tree
- Incomplete restoration of RO bind mounts options
Deprecated/removed
- Greedy mode of pagemap (non-root) caused dump to fail (disabled)
v. 2.2
New features
- Uninstall action in Makfilefile
- "Post-resume" added to action scripts
- Root task's PID in environment for action scripts
- C/R of
- Devconfs drop_gratuitous_arp and drop_unicast_in_l2_multicast
- * Serial ttys
Optimizations/improvements
- Lighter link-remaps restore on newer kernels
Fixes
- Race when restoring userns vs setting ns' maps
- Tasks with zero fds failed the dump
- Restore of TCP recv queue could fail due to kernel mem alloc constraints
- No errors were written to logs when launching helper (tar/iptables) app in userns restore
- User-mode dumped no memory pages sometimes
- Bind mounts considered not as bind sometimes
- Two mounts in the same directory blocked the dump
- Off-by in on /dev/tty{1,63} dumping
- Forking of cgroupns task was done with screwed clone flags
Deprecated/removed
- Greedy mode of pagemap dumping (on some kernels we do not support user-mode)
- Removed the --namespaces option
v. 2.1
New features
- Checking now classifies features to important/extra/experimental
- Ability to bring some disk files into images. See $source/scripts/tmp-files.sh
- C/R of
- Completed AIO requests
- Fallback gre and gretap net devices
Optimizations/improvements
- Code coverage collecting now works
- Use native rtnl library for netlink messages processing
- Using
--output - now results in stdout as log, not a file with the name "-"
- Signals are printed by names in logs
Fixes
Make tar generated tarbal with bad name- CG restore code lacked rollback in some places
- Error code from raw syscalls was treated with errors resulting in wrong
criu check reports
- Dumping task with HUGE amount of file descriptors failed
- Task could be stopped after pre-dump if respective option was used
- A /proc/pid directory from dead process conflicting with a new alive one could cause dump to fail
- Zombie from alien session/process group caused restore to fail
- CGroup fs was wrongly mounted in CGNS on restore
- Irmap scan was mis-checking devices numbers
- Use-after-free in irmap scan
- Btrfs bindmounts detection was mistaken due to 'subvol=' options met
- Propagation of mountpoint's shared groups was lost for propagated mounts
- Unaligned allocations of restore shared memory could result in codedumps when used by futexes
- Temporary mountpoints could result in spurious propagations
- When aborting the dump criu could crash on use-after-free objects
- Locking the network could stuck doing the DNS resolve
- Several build fixes
Deprecated/removed
- The images from criu prior to 0.4 are deprecated
- The
--namespaces option makes no sense and is also deprecated
- The
--ms option for check action is deprecated
v. 2.0
New features
- New code layout for sub-projects (e.g. Compel)
- Unprivileged dump
- Dump/check cpuinfo support for PPC
- Explorers for CRIT
- Added "post-setup-namespaces" to action scripts
- Added timeout for dump procedure (5 sec by default)
- Ability to override LSM profile on restore with CLI/RPC option
- External bind mounts can be fs-root mounts too
- Skip netns' internals on dump and restore (for Docker integration)
- Advanced support for external files
- C/R for
- Mode and uid/gid of cgroup files and dirs
- Freeze cgroup state (frozen/thawed)
- Task's loginuid and oom score
- Per-thread credentials
- Filter mode of seccomp
- Ghost file in removed directory
- Ghost files lutimes
- Binfmt-misc FS contents
- Netfilter conntracks and expectations
- Multi-headed cgroups
- CGroup namespaces (no nesting)
Optimizations/improvements
- Align parasite stack on 16 bits for correctness
- Compilation with native libc syscall wrappers and helpers
- Parasite code injection done via memfd system call
- Make vaddr to pfn conversion with one less syscall
- CRIT shows device numbers in "maj:min" manner
- CRIT shows mmap's status in verbose
- Docker files for builds on all supported arches
Fixes
- Absent readlink syscall on ARM (use readlinkat instead) could cause dump to fail
- Wrong argument to timer_create system call could cause restore to crash
- Extra tasks in freeze cgroup caused dump to fail/hand/crash
- Unaligned restore-time object allocations caused lock operations to fail
- Opened /proc/pid dir of dead task failed the dump
- Unaligned stacks caused criu to fail on aarch64
- Changed device numbers on restore side could cause random failures
- Fixes in mount points sharing/slavery/propagation restore
- Race between mntns creation and fds closing in different tasks could cause restore to fail
- Hard kernel limit on TCP repair recv queue restore could cause big queue restore to fail
- Unconnected dgram UNIX socket with data lost packets on restore
- CRIT didn't show IPC objects
- CRIT didn't convert IP addresses in images
- Logs from PIE code contained corrupted addresses and sizes
- Not loaded netfilter modules could cause dump/restore to stuck on dumping netlink socket
- Shared external mounts were restored with error
Security
- User-mode
- When checking for namespaces' CRIU entered userns with host creds
Deprecated/removed
- Completely removed 'show' action. Use CRIT instead.
v. 1.8
New features
- Ability to check CRIU features via RPC
- New zdtm.py test suite
- Pre-dump and pre-restore action scripts
- The "info" action in CRIT showing stats about image file
- More user-friendly output by CRIT
- Python API -- pycriu
- Ability to add custom paths to irmap scan
- C/R of
- read-only bind mounts
- IPv6 routes and iptables rules
- ip rules (it ip tool supports such)
- ignore_routes_with_linkdown netns devconf
- empty bridges in netns
- FILTER mode of seccomp
- IP_FREEBIND socket option
Optimizations/improvements
- Shared pie/non-pie .c files are built two times with proper flags
- VDSO code re-shuffled for better re-use between arches
- Failures of action scripts are reported in logs
- OpenVZ's VENET handling is tuned to fit the current kernel state
- Do not use hardcoded /dev/rts maj:min numbers
- Unsupported socket protocols are reported at expected place
- Slightly faster access to /proc files by using O_PATH open mode
- Improved page-server dump speed by keeping control over the Nagle algorithm
- Read pages.img in more optimal manner rather than page-by-page
- Less "Error"-s in logs, that actually don't lead to errors
- Slightly faster /proc/pid/status parsing
- Dead/live-locks on internal criu locks now emits a warning into logs
Fixes
- Page server flooded node with tw buckets during migration
- Turned off cgroups controllers weren't detected as such
- Netns sysctls from old images weren't properly restored
- Running process could be mistakenly stopped after --leave-running dump
- Helper processes run by CRIU produced fake error messages in logs
- Error code from sigaction restore could be missed
- Several potential buffers overruns due to missed '\0' after strcpy-s existed
- Killed processes after dump survived in zombie state for some time holding PIDs and resources
- If task had MANY children, the latter could be skipped on dump
- Task dying while being frozen could fail the dump
- On Aarch64 the upper limit for user memory was not properly detected sometimes
- Guess for TCP buffer max segment size was too optimistic (could fail the restore on low-mem machines)
- CRIT didn't decode userns images
- Ghost files were left in the FS tree after failed restore (blocking the next restore attempt)
- Some log messages from pie code were lost
- Some net/ipc/uts sysctls failed to restore in userns
- Move tasks int cgroups failed in userns
- Unsupported filesystems silently failed the dump
- External tmpfs (and some other) mounts generated tarballs with their contents
- Privately mapped files were picked from wrong mount namespace
- Controlling tty could be restored on wrong tty end
- Tmpfs mount of sub-namespace was restored from wrong image file
- Potential stack overflow in libcriu
- Partially-restored tasks could be left after failed restore
- In-container TCP connection sometimes failed to restore
- Race in sending SIGSTOP vs dump might cause dump to fail
- Post-restore actions could generate stats files in wrong directories
- Freeze-cgroup didn't take sub-cgroups' tasks into account
- Tentative state in IPv6 sockets binding prevented socket from being bound immediately
- Restoring from images with files pointing to /proc file of dead tasks could crash
- Tasks with STOP in queue (i.e. -- not yet stopped) were CONT-ed in case of --leave-running dump
- Stopped task with one more STOP in queue caused dump to stuck
- If parent task left the MNT namespace it created for children restore could BUG()
- Link-local IPv6 addresses sometimes failed to bind() at restore
Security
- Service run as root could allow users to violate ptrace policies
- Service run as root could give users access to privileged files and directories
v. 1.7.2
Fixes
- Mounting container root on restore could sometimes switch to wrong root path
- The slave/shared option for CT root was lost on restore
- Duplicate slave mount points could appear on restore
- Fanotifies (inotifies) could be restored in wrong mount namespace (though on correct inode)
- Fanotifies (inotifies) on bind-mount-ed tmpfs file could fail the dump
- Kernel threads found in tree (OpenVZ containers case) blocked the dump
- Flat user namespace (0; \infty) restore failed
- Off-by-one in unix socket name handling
- IPC objects' UIDs and GIDs were not treated as userns ones
- Rcv and Snd buffers for sockets grew 2 times on restore
v. 1.7
New features
- More flexible CGroups managing on restore
- Support for seccomp strict mode
- Support for stream unix sockets inheritance
- Support uid/gid-restricted mounts in userns
- Support deleted bind-mounts
- Freezer cgroups can be used on dump to freeze fast-spawning processes
- Ability to specify maximum ghost file size
- OverlayFS support
- Support relative unix sockets' bind paths
- In libcriu
- New set of calls using non-global opts
- Ability to pass existing connection to service
- Ability to start criu in swrk mode for all requests
- Arch-specific improvements
- Altivec and PSX support for PPC
- Small PIE loader
- Preparations for 32-bit x86
Optimizations/improvements
- Temporary proc mountpoint is mounted with nosuid, noexec and nodev
- Less memory copies when preparing restorer binary
- CRIT action "show" for less keystrokes on common use-case
- Fsnotify log messages now use hex everywhere :)
- CRIT output doesn't mix fields any more
Fixes
- CRIU binary couldn't be installed independently from man pages
- Root dir ignored in install: target
- Bug in restoring PPC floating point register
- SYSVIPC shmem was not attached on restore with PPC
- AIO ring ID was erroneously close()-d
- After dump+kill tasks remained in zombie states
- Race in zombies vs proc proxy tasks deaths resulted in restore spurious failure
- Restore got stuck when CRIU was called with blocked SIGCHILD
- Wrong page size value could be used on some ARM compilations
- Potential memory corruption when restoring an LSM profile
- Opened /dev/kmsg in WRONLY mode failed the restore
- Weird paths on tmpfs caused tar to fail
- Temporary cgroup mount set (cgyard) got propagated into the host tree
- Restore of inherited shared pipe failed
- Spaces, tabs and backslashes in mountpoints' paths caused dump to fail
- Tmpfs mounted with empty source caused dump to fail
- The criu.pc file contained bad version when built from tarball
- Deprecated -n option found in docs
- On aarch64 the maximum virtual address available for user-space was wrongly hardcoded
v. 1.6.1
New features
- Support for relative paths for unix sockets
Fixes
- Crash when restoring netns from older images
- Race between unix sockets' connect and listen may cause restore to fail
- Multiple unix datagram clients restored server queue multiple times
v. 1.6
New features
- PowerPC 64bit LE support
- Makefile.local for 3-rd party build rules
- Ability to "enable" filesystem on dump (--enable-fs)
- Ability to skip mountpoint on dump (--skip-mnt)
- Prepare to deprecate "criu show" command
- External mounts auto-detection
- External siblings resolving
- External sharing resolving
- /dev/tty (current terminal) support
- Netdev and netns (all/default) confs C/R
- Images v1.1 with extra magic at head
- Support fusectl (only ctl) mountpoint
- Sub-version format is now as of git-describe
- Apparamor labels C/R support
Optimizations
- Empty image files are not generated in image dir
- /proc/pid/fd/locks support for faster and non-intrusive locks dump
Fixes
- Cscope scanned symlinks on make tags
- Compilation with clang failed
- Improper PAGE_SIZE constant was used on Aarch64
- Selinux blocks attempt to inject parasite w/o any reasonable message
- Error code masked on some error paths
- O_APPEND files' changed size aborted restore
- Errno value could be overwritten by logging
- Mount namespace w/o /tmp could not be dumped
- Stats file generated in wrong dir sometimes
- MS_STRICTATIME mountpoint option was dropped on dump
- Read-only tmpfs mount failed to restore
- Some files were put into wrong places upon install target
- Service couldn't be enabled via systemd ctl after manual installation
- Parent's /proc/self files could be accessed by criu processes on restore
- When meeting unknown image file CRIT exited with exception instead of printing sane error message
v. 1.5.2
Fixes
- Mutli-threaded tasks restored with error when --restore-sibling (Docker and LXC cases)
- Service (and swrk) couldn't receive too big RPC messages
v. 1.5.1
New features
- Inheriting FDs now work in "swrk" RPC mode
- Restored pid is reported in post-restore RPC notification
Fixes
- Uninitialized ss in sigframe causes C/R failures on 4.0 kernel
- Cgroups' properties are initialized too late on restore
- Cgroups' destruction isn't performed in non detached mode
- Cgroups' destruction can fail on error paths
v. 1.5
New features
- CRIT tool
- Ability to request CPU compatibility on instructions level only
- C/R of empty AIO rings
- More detailed errno report via RPC
- Per-feature "criu check"
- Inheriting FDs on restore
- Ability to automatically move veth device to host-side bridge on netns restore
- VT terminals support
- More user namespaces C/R stuff
Optimizations
- TCP send queue is restored in the maximal portions allowed by the kernel
- Pre-loading sock-diag modules now happens in a more elegant way
Fixes
- Multi-threaded tasks on 64bit ARM could segfault upon restore
- When doing "check" CRIU could leave un-killed piggie task
- The --cpu-cap option argument was parsed with errors
- Incorrect handling of --cpu-cap fpu compatibility mode on restore
- Criu ignored trailing CLI arguments that resulted in usage confusions
- Irmap hints didn't include common "/" path
- When run per user request, CRIU left log and pid files belonging to root
- Mappings on AUFS could be looked up on wrong mount point
- Fixed compilation on Centos6.5
- Wrong /proc was used when reading the list of FDs to close on restore
- Race in restoring TCP established and listening sockets results in failed bind() on the latter
- Legacy ttys errorneously treated as unix98
- TTY pairs slavery setup could pick wrong peer
- For user-dump the log and pid files still belonged to root
- Task could die while being frozen thus causing dump to fail or save wrong task state
- Failures in mount points validation and sharing resolving didn't abort the dump (error arose on restore)
v. 1.4
New features
- Dump and check cpuinfo. Needed to make sure CPU is capable to run the images after restore, e.g. during live migration
- Initial support for user namespaces
- Use memfd to restore shared memory segments
- New (slightly faster) API for mm stuff restore via prctl
- [UG]ID-s are dumped from parasite, not from /proc files
- The docker_cr.sh script to show how Docker container C/R should (will) look like
- New API for writing plugins (old one is still possible)
- Service workers change their title to better look in ps output
- Ability to feed socket for pre-dump and page-server in swrk mode
- Page-server can auto-bind its port
- Ability to perform several actions during one connection to RPC service
- C/R of opened /proc/$pid/foo files of dead tasks
- C/R of /dev/console
- C/R of virtualized devtmpfs (openvz and future upstream kernels)
- C/R of empty mqueue fs (posix message queues)
- C/R of shared bind-mounts
Optimizations
- BFD engine
- Faster that glibc's FILE * buffered read from /proc files
- Buffered image files IO
- Faster parasite/restorer unload
- Use HW breakpoints
- Less ptrace GETREGS calls sometimes
- Wake pie after sending the FINI command to socket
- Merged some pairs of images into one
- eventpoll and -tfd
- inotify and -wd
- fsnotify and -mark
- Less setns()-s on dump is much faster on older kernels
- Faster access to /proc/self files -- cached fd of /proc/self and openat(this_cache)
Fixes
- Sibling restore mode didn't set up CRIU signals properly
- Unpredictable sibling/child root task restore. Fixed with explicit CLI option
- Validation for leaf mount points was skipped
- Mount options were corrupted on dump, which resulted in errors bind mounts detection
- Uninitialized properties of some cgroups prevented moving tasks into them (e.g. empty cpuset masks and low memcg limit)
- File locks could belong to task with different pid (inherited on fork) blocked the dump
- Bogus error printed in logs about SIGCHLD catch (was caused by thread dump using traps)
- Irmap engine accessed freed root_task on pre-dump
- Restore of net namespace could always fail (pid mismatch on fork) if kernel thread was created on netns setup
- Cgroups service descriptor was closed too early and failed restore
- Auto-loaded *diag modules caused audit netlink socket to contain data on dump (dump fails in this case)
- The "(deleted)" prefix accumulated in unlinked files while doing C/R
- The devpts filesystem and ptmx file were only dumped when found on /dev/pts and /dev respectively
- Data in netlink socket and fanotify was lost after C/R (now dump is aborted if data found in it)
- Fanotify mark was restore in different mount namespace
- Images were writable by group. Not secure when user-dump was requested
- Rootfs has parent id equal to self. CRIU didn't expect this and failed the dump
- Shared mount of the --root path failed the restore
- Absence (e.g. not compiled in) of any namespace in the kernel failed the dump
- Page-server incremental dump didn't detect new tasks properly and failed the stage
- Big TCP queues sometimes failed to get restored
- Incremental pre-dump could lose track of memory changes by task
v. 1.3.1
Fixes
- Sibling restore mode didn't set up CRIU signals properly
- Unpredictable sibling/child root task restore. Fixed with explicit CLI option
- Validation for leaf mount points was skipped
- Mount options were corrupted on dump, which resulted in errors bind mounts detection
v. 1.3
New features
- TimerFD support
- VVAR area (newer kernels' part of VDSO) support
- CGroups hierarchies support
- AUFS support (for Docker)
- PDeathSig support
- Check for opened file's size on dump and restore is the same
- Ability to restore tasks as children using libcriu (
criu_restore_child)
- Add pkgconfig file for libcriu
- CRTOOLS_IMAGE_DIR variable available in action scripts
Optimizations
- Merged images with pending signal into core
- Per-task images with file locks are merged into one big image
- Smaller tasks orchestration memory area on restore
- Sigactions are inherited on restore when possible, not overwritten
- ZDTM suite now executes tests in parallel
Fixes
- Dump failed if robust lists were off
- Link remaps on tmpfs mounts were not dumped
- Non root tasks with custom groups couldn't dump its peers (Security)
- Opened and unlinked FIFOs, dirs and devices were restored as regular files
- Files opened from alien mount namespace were restored in the local one
- Link remap name sometimes was generated with error
- Opened and removed cwd couldn't be restored
- Sysctl kernel.msgmni was overwritten by subsequent auto_msgmni
- Library and RPC APIs didn't match the CLI one
- Some external mounts were constantly "postponed" and never got mounted
- The self.mm_dumpable prlctl value of 2 caused restore to fail
- Errors when writing sysctls with tail \n
- The
criu show printed nested repeated fields corrupted
- Dump stats were initialized with garbage
- Restore sometimes stuck on waiting for inet socket port bind
- Spurious SIGHUP when restoring slave ttys
- Restore wasn't aborted if sub-task failed early
v. 1.3-rc2
New features
- Native (w/o plugins) c/r of external bind mounts
- C/R of the info in which cgroups tasks live
- C/R of task's dumpable flag
- Dump pstore, securityfs, fusectl and debugfs mountpoints
Fixes
- VDSO was searched on stack's guard page
- Mount namespace w/o /proc mount blocked the restore
- Several misses in searching for COW VMA resulted in sub-optimal pages sharing on restore
- FIFO-s path was restored in wrong mount namespace
- Mountpoint fsnotify could be restored on a bind-mount
- One tmpfs mounted several times was dumped several times
- Bind-mount's root path of the top mount was calculated with error
- Fix device number calculation out of major:minor on some distros
- Devpts mount options got lost on dump
- Page-pipes grew endlessly resulting in dump failures on big VMAs
- IO and PF mappings were tried to be dumped
- Two merged MAP_GROWSDOWN VMAs got dumped with overlapping guard page
- Too small shared area was used to fetch tasks mappings that resulted in failed dump of huge mappings
- Many fixes in build system
- Zdtm's COW test sometimes ignored COW failures
v. 1.3-rc1
New features
- AArch64
- Multiple mount namespaces
- FPU state restore control
- Restore old FPU state on newer CPUs
- Ability to ignore FPU restoration
- Support stopped multi-threaded tasks
- CRIU now can execv() other binary right after restore is complete
- Inode-reverse mapping can be enforced to allow live-migration with FS copying
- Gold linker can now be used to compile CRIU
- "Berserker" test to check CRIU scalability
- Punch pages from mem images on restore (optimizes live-migration)
Optimizations
- Batched deduplication of memory images
- Packed rlimits into core image
- Packed timers into core image
Fixes
- Bad checks for
kcmp() ret codes resulted in errors in file sharing detection
- Multiple mmaps of same files with different flags blocked the restore
- Integer overflow in huge mapping restore caused restoration failure
- devpts's
newinstance option was lost during dump
- Subsequent dump could try to find old mem dump for newly forked task
- Bad detection of overmounted mountpoints on fsnotify restore
- Page-server could read partial message and failed
- Errors in dumping of two subsequent anon VMAs in some cases
- Irmap mis-compared devices for disk FSs
- TMPFS handles always change during dump/restore
- Pre-dump sometimes hangs on FIFOs
- Post-restore script fails too late (if does it)
v. 1.2
New features
- Performance improvements
- Shared entries in reg-files image
- Less accesses to
/proc/$pid/map_files links
- Cache for
/proc/$pid/pagemap reads
- VDSO page is seeked only in anonymous mappings
- Task's auxv is read in one call
- Merged mm and vma image files for better packing
- NFS inodes' path resolution (for fsnotify) cache
- One
readlink() call when checking anon inodes
- Don't dump kernel's zero-page
- Parse fast
/proc/self/maps when searching for hole for restorer
- A bit faster write into image files with
writev()
- Library versioning
- RPC API got closer to CLI
- New "post-restore" call in action scripts
- Logrotate rules file
- Default log file for service when starting via systemd
Fixes
- A lot for ARM cross-compile
- Fsnotifies dumping didn't work on NFS
- Images auto-deduplication only worked one level up
- Packet socket ID was treated as file-descriptor and close()-d
- Badly counted pages stats on restore
- Linked remap name conflict when dump and restore on NFS
- Sporadic failures in memory draining due to huge pipes used
- Broken
criu show of repeated fields
- Failure to open mountpoint in foreign pid namespace
- Unlinked bound unix socket dump error
- Small memory leak when writing to incremental image(s)
- Restoring fsnotify for links results in ELOOP
- Host's PATH is not suitable when execv-ing tar/ip/iptable to restore namespace (workaround, proper fix will be in 1.3)
- Using subdirs in log file name via RPC breaks security
v. 1.1
Fixes
- Errors from memory dumping are not handled resulting in corrupted dumps
- EOF detection in stacked images is done with error
- Stacked images don't work on non-shared FS (missing pagemap-s)
v. 1.1-rc2
Fixes
- Crash in
criu check
- RPC check always fail on 3.11 kernel
- Failed fork() didn't abort restore
- Dump fail not reported via RPC
- RPC client disconnect wasn't handled
- Page server could connect to self for writing images
- Hang on pre-dumping task livig in net-namespace
- VDSO page mis-handle on pre-dump
- FPU state loss on pre-dump
- Memory tracking turns ON w/o request
- Various fixes (and improvements) in build system
v. 1.1-rc1
New features
- libcriu.so -- wrapper library for RPC clients
- Plugins
- External unix sockets
- External bind mounts
- External net devices
- Unknown file types
- Images deduplication in incremental dumps
- Integration with systemd
- Filtering of
criu show output
|
Note: The API defined in the first two items above may change after -rc1
|
Fixes
- Errors in unlinked files/sockets detection on BTRFS
- NFS silly-rename files are not treated as unlinked
- Freezer fail to seize quickly forking/pthread_create-ing tasks
- Extra stop signal queued for stopped tasks after pre-dump
- Wrong dying task state detection
- Lost RPC dump response
- Crash when reporting restore error via RPC
- Negative return code into shell
- Tasks left in wrong states after failed dump
- A little bit more verbose check action
- Coverity checks fail here and there
v. 1.0
Fixes
- After --leave-running linked remaps were not cleaned
- TCP was left locked after --leave-running
- Weak criteria in memory COW detection
- Private mapping's premmapped address overwrote file ID
- Restorer memory could overlap with timers/signals arrays
- RPC worker reused options from service task
- Suboptimal memory utilization by restorer arguments
- TCP unsent/unacked data boundary was lost
- Wrong dev_t decoding on 64 bit
- Unpredictable daemons (service and page-service) working dir
- Parasite stack could be corrupted by its arguments
- Error from exe link restore was ignored
- Artificial small limit on the number of shared memory segments to restore
- Bug in ARM VFP restore
- VDSO proxy was unmapped at the very end of restore
New features
- -W option to specify working dir
- CHECK request in RPC
- Optimized headers
- More info in logs about undumpable files
- More comments about tricky dump/restore places
- Generic memory allocation for restorer
v. 0.8
New features
- RPC service
- Ability to work from non-root user (via +s bit)
- Handle stopped tasks
- Restore tasks' root path
- Dump and restore net ns iptables configuration (w/o conntracks)
- Support for external net devices in netns (e.g. openvz venet)
- Support CORK and NODELAY TCP options
- SEQPACKET unix sockets support
Fixes and improvements
- Unload restorer blob after restore
- Fixes and enhancements in
criu show
- Fix in unsorted inotify wd restoring
- Fixed trimmed messages in parasite transport
- Fine-grained pgrps restore
- Fix in large TCP buffers restore
- Fixed buffer overflow in IPC ns dumping
- Fix in early page server connection close on pre-dump
- Fixed race in handling aborted parasite blob
- Fixed lost unmapped criu vmas in restore
- Fixes in parsing devices in /proc/pid/maps and /proc/locks
- Fixed snd/rcv buf sockoptions restore
- Enhanced logging in parasite
v. 0.7
New features
- TUN devices support (requires patched kernel)
- Alternative stack (sigaltstack) C/R
- Shared and master/slave mounts support
- Restore statistics
- Net device address C/R
- Safer and simpler asynchronous parasite
- Fixes in
- Stack guard page dump/restore
- Page server communications
- COW mappings restore
- Zombies restore
- Aborting failed restore
- TCP connection restore (some require patched kernel)
- Semi-closed Unix sockets with data dump/restore
- etc.
v. 0.6
New features
- Cross-compiling support
- Ready to accept images from OpenVZ RHEL6-based kernel
- Posix CPU timers C/R
- Asynchronous parasite
- Self-heal dumpee in case of crtools crash
- Memory changes tracking (requires patched kernel)
- Incremental backups
- Pre-dump task command for smaller freeze time
- Dump statistics
- Return-arg for remote syscalls execution
- Improved "show" output
- Opened /proc/PID/ns/* files C/R
- Daemon mode for page-server
- Build-time features test (makes it possible to build on different distros)
v. 0.5
New features
- C/R netlink sockets
- C/R pending signals
- New format for memory dumps
- Incremental dumps (preliminary)
- Ability to directly send memory dumps on remote host (for faster live migration)
- Images showing enhancements
- Kernel features checking enhancements
- Lots of BUGs fixed
- Renamed
crtools to criu (both binary and source package names)
v. 0.4
crtools-0.4.tar.bz2
Released: 20 Feb 2013
GIT tag: v0.4
New features
- ARM port
- Remote syscall execution
- C/R of
- FPU state
- File locks (basic support)
- Rlimits
- FANotify descriptors
- Shared fdtable (table of file descriptors)
- Tasks' umask
- Pipe buffer size
- Unix sockets' credentials
- TCP time-stamp offset (allows to migrate a TCP socket, requires custom kernel)
- Automatic namespaces detection (
--namespace option is deprecated)
Internal
- Build system rework
- Dumping/restoring memory pointers standardized (while doing ARM port)
- Ability to collect coverage (gcov, reported here)
- 32/64-bit problems mostly resolved (while doing ARM port)
v. 0.3
crtools-0.3.tar.bz2
Released: 11 Dec 2012
GIT tag: v0.3
New features
- C/R of shell jobs
- Handle files, that are opened and unlinked, but some other hard link exists
- More SOL_SOCKET socket options. Two most interesting are:
- Binding socket to device (
SO_BINDTODEVICE)
- Per-socket packet filter (
SO_ATTACH_FILTER)
- Shutdown state of a socket
- Task scheduler parameters (nice, policy and prio)
- Properly handle COW pages
- Memory mapped packet sockets (with SOL- options this is enough for tcpdump tool support)
- Complementary groups (
getgroups(2))
- Various mapping features, configured with
madvise(2)
- Tune TCP sockets support to work with IPv6 sockets
- Closed (or not yet connected/bound) unix sockets
Bug fixes and improvements
- Print timestamps in log files
- Restructure source tree a little
- PIE stuff in separate dir
- Arch-specific code in arch/<Arch>/ directory
- Nicer
show -c output for pages/pipes/etc contents
- Fixes in help text
- Save and restore blocked signals mask for threads
- Don't dump (and warn about it) corked UDP sockets
- Fixed livelocks when restoring too big socket queues
- Refresh info about sockets, that might have changed between diag dump and actual process freeze
- Lost file params and socket options for some unix sockets
- Fix inotify on opened and unlinked files
- Fix shared anonymous memory detection
- Properly close all files before restoring them
- Fixes in running external tools (ip/tar)
- Several fixes in control tty restoring code
- RT signals handlers dumped and restored
v. 0.2v. 0.1
