Xsave

Summary

XSAVE stands for similar x86 instruction xsave which places extended processor state into a memory area. The saving can be initiated by any userspace application at any moment and size of the memory frame depends on processor features and may vary between different models. Thus if checkpoint and restore are done on different processors the next call to xsave may corrupt memory if sizes mismatch.

Helpers

There are several helpers we will refer on in this page

static inline void native_cpuid(unsigned int *eax, unsigned int *ebx,
				unsigned int *ecx, unsigned int *edx)
{
	/* ecx is often an input as well as an output. */
	asm volatile("cpuid"
	    : "=a" (*eax),
	      "=b" (*ebx),
	      "=c" (*ecx),
	      "=d" (*edx)
	    : "0" (*eax), "2" (*ecx)
	    : "memory");
}

static inline void cpuid(unsigned int op,
			 unsigned int *eax, unsigned int *ebx,
			 unsigned int *ecx, unsigned int *edx)
{
	*eax = op;
	*ecx = 0;
	native_cpuid(eax, ebx, ecx, edx);
}

static inline void cpuid_count(unsigned int op, int count,
			       unsigned int *eax, unsigned int *ebx,
			       unsigned int *ecx, unsigned int *edx)
{
	*eax = op;
	*ecx = count;
	native_cpuid(eax, ebx, ecx, edx);
}

Frame size

Run cpuid(0x1, &eax, &ebx, &ecx, &edx) and bits 26 and 27 are both set in ecx if xsave is supported (strictly speaking bit 27 is reserved for operating system which can clear it to indicate that instruction is disabled).

After that we can fetch maximal frame size which applications may use via cpuid_count(0xd, 0, &eax, &ebx, &ecx, &edx), in result ebx will contain the size to keep currently enabled components of the frame and ecx will keep the value of maximal frame size. The maximal here means the size needed when all components are enabled (OS may disable some of components).

Enumerating frame components

To enumerate which components of the frame are enabled execute cpuid_count(0xd, 0, &eax, &ebx, &ecx, &edx). Each component will have bit set to 1 in 64 bit mask eax + ((uint64_t)edx << 32) if enabled.

Current list of known components are the following (numbers are the bit position):

• 0: x87 floating point registers
• 1: SSE registers
• 2: AVX registers
• 3: MPX bounds registers
• 4: MPX CSR
• 5: AVX-512 opmask
• 6: AVX-512 Hi256
• 7: AVX-512 ZMM_Hi256
• 8: Processor Trace
• 9: Protection Keys User registers
• 10: Hardware Duty Cycling