Xen project Mailing List

On May 10, 2012 3:40 PM, "Jeff Law" <law@xxxxxxxxxx> wrote:
>
> On 05/09/2012 11:38 AM, Konrad Rzeszutek Wilk wrote:
>>>
>>>
>>> And finally one always has to keep in mind that there is this nice
>>> glibc bug in that in some versions it is failing to look at CPUID.OSXSAVE
>>> when trying to determine whether AVX or FMA is available.
>>
>>
>> It has this:
>> 146
>> 147 if (__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx& bit_AVX)
>>
>> 148 {
>> 149 /* Reset the AVX bit in case OSXSAVE is disabled. */
>> 150 if ((__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx& bit_OSXSAVE) != 0
>> 151&& ({ unsigned int xcrlow;
>>
>> 152 unsigned int xcrhigh;
>> 153 asm ("xgetbv"
>> 154 : "=a" (xcrlow), "=d" (xcrhigh) : "c" (0));
>> 155 (xcrlow& 6) == 6; }))
>>
>> 156 __cpu_features.feature[index_YMM_Usable] |= bit_YMM_Usable;
>> 157 }
>>
>>
>> And when I ran a little silly C program (attached) to probe the CPUID flags, I got:
>> /root/test-xsave
>> SSE3 CMOV
>> AVX XSAVE
>> Trying XGETBV
>> Illegal instruction (core dumped)
>>
>> Which would imply that the OSXSAVE is not set (but Fedora Core 17 64-bit PV guest
>> still crashes on that machine) - which means that in multi-lib the bit_YMM_Usable
>> is _not_ set. But then looking at the sources I see:
>>
>> # ifdef USE_AS_STRCASECMP_L
>> 102 ENTRY(__strcasecmp)
>> 103 .type __strcasecmp, @gnu_indirect_function
>> 104 cmpl $0, __cpu_features+KIND_OFFSET(%rip)
>> 105 jne 1f
>> 106 call __init_cpu_features
>> 107 1:
>> 108 # ifdef HAVE_AVX_SUPPORT
>> 109 leaq __strcasecmp_avx(%rip), %rax
>> 110 testl $bit_AVX, __cpu_features+CPUID_OFFSET+index_AVX(%rip)
>> 111 jnz 2f
>> 112 # endif
>>
>> which would imply that the AVX bit is sampled here instead of the
>> YMM one.
>
> [ ... ]
> I think Uli's position is that this code only uses AVX encodings, but not the YMM registers and thus the right check is for AVX.
>
> That doesn't make sense to me given the text under availability and support here:
>
> http://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions/
>
> According to my reading AVX can only be used if the hardware supports AVX *and* the OS supports XSAVE. The only weasel language is "To use the Intel AVX extensions reliably in most settings ..." Which Uli might be relying upon for his position.
>
> Ironically, the code in init-arch used to look like:
>
>
> if (__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx & bit_AVX)
> {
>
> /* Reset the AVX bit in case OSXSAVE is disabled. */
> if ((__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx & bit_OSXSAVE) == 0
> || ({ unsigned int xcrlow;
> unsigned int xcrhigh;
> asm ("xgetbv"
>
> : "=a" (xcrlow), "=d" (xcrhigh) : "c" (0));
> (xcrlow & 6) != 6; }))
>
> __cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx &= ~bit_AVX;
> }
>
>
> Which I think would have done the right thing. Uli changed it to the form you quoted just 2 hours after installing the version I quoted.

Sadly no as it would have executed the xgetv instruction. Since the first part of the boolean logic returns false.
>
> If i'm going to make the claim Uli is wrong, some clarification from Intel would be appreciated.
>
>
> jeff
>

On May 10, 2012 3:40 PM, "Jeff Law" <law@xxxxxxxxxx> wrote:

On 05/09/2012 11:38 AM, Konrad Rzeszutek Wilk wrote:

And finally one always has to keep in mind that there is this nice
glibc bug in that in some versions it is failing to look at CPUID.OSXSAVE
when trying to determine whether AVX or FMA is available.

It has this:
146
147 if (__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx& bit_AVX)
148 {
149 /* Reset the AVX bit in case OSXSAVE is disabled. */
150 if ((__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx& bit_OSXSAVE) != 0
151&& ({ unsigned int xcrlow;
152 unsigned int xcrhigh;
153 asm ("xgetbv"
154 : "=a" (xcrlow), "=d" (xcrhigh) : "c" (0));
155 (xcrlow& 6) == 6; }))
156 __cpu_features.feature[index_YMM_Usable] |= bit_YMM_Usable;
157 }

And when I ran a little silly C program (attached) to probe the CPUID flags, I got:
/root/test-xsave
SSE3 CMOV
AVX XSAVE
Trying XGETBV
Illegal instruction (core dumped)

Which would imply that the OSXSAVE is not set (but Fedora Core 17 64-bit PV guest
still crashes on that machine) - which means that in multi-lib the bit_YMM_Usable
is _not_ set. But then looking at the sources I see:

# ifdef USE_AS_STRCASECMP_L
102 ENTRY(__strcasecmp)
103 .type __strcasecmp, @gnu_indirect_function
104 cmpl $0, __cpu_features+KIND_OFFSET(%rip)
105 jne 1f
106 call __init_cpu_features
107 1:
108 # ifdef HAVE_AVX_SUPPORT
109 leaq __strcasecmp_avx(%rip), %rax
110 testl $bit_AVX, __cpu_features+CPUID_OFFSET+index_AVX(%rip)
111 jnz 2f
112 # endif

which would imply that the AVX bit is sampled here instead of the
YMM one.

[ ... ]
I think Uli's position is that this code only uses AVX encodings, but not the YMM registers and thus the right check is for AVX.

That doesn't make sense to me given the text under availability and support here:

http://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions/

According to my reading AVX can only be used if the hardware supports AVX *and* the OS supports XSAVE. The only weasel language is "To use the Intel AVX extensions reliably in most settings ..." Which Uli might be relying upon for his position.

Ironically, the code in init-arch used to look like:

if (__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx & bit_AVX)
{
/* Reset the AVX bit in case OSXSAVE is disabled. */
if ((__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx & bit_OSXSAVE) == 0
|| ({ unsigned int xcrlow;
unsigned int xcrhigh;
asm ("xgetbv"
: "=a" (xcrlow), "=d" (xcrhigh) : "c" (0));
(xcrlow & 6) != 6; }))
__cpu_features.cpuid[COMMON_CPUID_INDEX_1].ecx &= ~bit_AVX;
}

Which I think would have done the right thing. Uli changed it to the form you quoted just 2 hours after installing the version I quoted.

If i'm going to make the claim Uli is wrong, some clarification from Intel would be appreciated.

jeff

Re: [Xen-devel] xsave=0 workaround needed on 3.2 kernels with Xen 4.1 or Xen-unstable.