Xen project Mailing List

Re: [Xen-devel] [PATCH linux 2/8] xen: introduce xen_vcpu_id mapping

On 29/06/16 17:19, Vitaly Kuznetsov wrote:

Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> writes:

> Andrew Cooper <andrew.cooper3@xxxxxxxxxx> writes:
>

>> On 29/06/16 13:16, Vitaly Kuznetsov wrote:

>>> Andrew Cooper <andrew.cooper3@xxxxxxxxxx> writes:
>>>

>>>> On 28/06/16 17:47, Vitaly Kuznetsov wrote:

>>>>> @@ -1808,6 +1822,8 @@ static int xen_hvm_cpu_notify(struct notifier_block *self, unsigned long action,
>>>>>  	int cpu = (long)hcpu;
>>>>>  	switch (action) {
>>>>>  	case CPU_UP_PREPARE:
>>>>> +		/* vLAPIC_ID == Xen's vCPU_ID * 2 for HVM guests */
>>>>> +		per_cpu(xen_vcpu_id, cpu) = cpu_physical_id(cpu) / 2;

>>>> Please do not assume or propagate this brokenness.  It is incorrect in
>>>> the general case, and I will be fixing in the hypervisor in due course.
>>>>
>>>> Always read the APIC_ID from the LAPIC, per regular hardware.

>>> (I'm probbaly missing something important - please bear with me)
>>>
>>> The problem here is that I need to get _other_ CPU's id before any code
>>> is executed on that CPU (or, at least, this is the current state of
>>> affairs if you look at xen_hvm_cpu_up()) so I can't use CPUID/do MSR
>>> reads/... The only option I see here is to rely on ACPI (MADT) data
>>> which is stored in x86_cpu_to_apicid (and that's what cpu_physical_id()
>>> gives us). MADT also has processor id which connects it to DSDT but I'm
>>> not sure Linux keeps this data. But this is something fixable I guess.

>>
>> Hmm yes - that is a tricky issue.
>>
>> It is not safe or correct to assume that xen_vcpu_id is APICID / 2.
>>
>> This is currently the case for most modern versions of Xen, but isn't
>> the case for older versions, and won't be the case in the future when I
>> (or someone else) fixes topology representation for guests.
>>
>> For this to work, we need one or more of:
>>
>> 1) to provide the guest a full mapping from APIC_ID to vcpu id at boot
>> time.

>
> So can we rely on ACPI data? Especially on MADT and processor ids there?
> I think we can always guarantee that processor ids there match vCPU
> ids. If yes I can try saving this data when we parse MADT.
>

To explain better what I'm trying to suggest here please take a look at
the attached patch. If we can guarantee long term that ACPI id always
equals to Xen's idea of vCPU id this is probably the easiest way.

-- 
  Vitaly

The code in hvmloader which sets up the MADT does:

    for ( i = 0; i < hvm_info->nr_vcpus; i++ )
    {
        memset(lapic, 0, sizeof(*lapic));
        lapic->type    = ACPI_PROCESSOR_LOCAL_APIC;
        lapic->length = sizeof(*lapic);
        /* Processor ID must match processor-object IDs in the DSDT. */
        lapic->acpi_processor_id = i;
        lapic->apic_id = LAPIC_ID(i);
        lapic->flags = (test_bit(i, hvm_info->vcpu_online)
                        ? ACPI_LOCAL_APIC_ENABLED : 0);
        lapic++;
    }

So relying on the acpi_processor_id does look to be reliable. That code hasn't changed since 2007, and that was only a bugfix. I would go so far as to say it is reasonable for us to guarantee this in the guest ABI.

Jan - thoughts?

~Andrew

_______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel