Re: [Xen-devel] FW: VT-d async invalidation for Device-TLB.

["Zhang, Yang Z" <yang.z.zhang@xxxxxxxxx>]

Jan Beulich wrote on 2015-06-16:
>>>> On 16.06.15 at 05:07, <yang.z.zhang@xxxxxxxxx> wrote:
>> Jan Beulich wrote on 2015-06-15:
>>>>>> On 13.06.15 at 16:44, <quan.xu@xxxxxxxxx> wrote:
>>>>> On 12.06.15 at 14:47, <JBeulich@xxxxxxxx> wrote:
>>>>>>>> On 12.06.15 at 04:40, <quan.xu@xxxxxxxxx> wrote:
>>>>>> I tested it by a Myri-10G Dual-Protocol NIC, which is an ATS device.
>>>>>> for an invalidation:
>>>>>>  By sync way, it takes about 1.4 ms.
>>>>>>  By async way, it takes about 4.3 ms.
>> 
>> It is a typo for the time: should be 1.4 us and 4.3 us.
>> 
>>>>> 
>>>>> What's the theory on why this is? After all, it shouldn't matter
>>>>> how the completion of the invalidation gets signaled.
>>>>> 
>>>> 
>>>> Let me introduce more about how I get these test data.
>>>> 
>>>> For sync way, I get the time by NOW() macro, when I update Queue Tail
>>>> Register  (qinval_update_qtail()). Then get time by NOW() macro in
>>>> current spinning when it has wrote the value in the Status Data field
>>>> to the address specified in the Status Address. The difference of
>>>> these 2 value is the time of an sync invalidation.
>>>> 
>>>> 
>>>> For async way, as the previous email, I have introduced the IF bit
>>>> of Invalidation Wait Descriptor.
>>>> (IF: Indicate the invalidation wait descriptor completion by
>>>> generating an invalidation completion event per the programing of
>>>> the Invalidation Completion Event Registers.) I have implemented
>>>> an interrupt for invalidation completion event.
>>>> Also I get the time by NOW() macro, when I update Queue Tail
>>>> Register (by qinval_update_qtail()).
>>>> Then get time by NOW() macro in invalidation completion event handler.
>>>> The difference of these 2 value is the time of an async invalidation.
>>> 
>>> Okay, thanks for the explanation. As this includes the time it
>>> takes to deliver and (partly) handle the interrupt, the difference
>>> is of course within what one would expect (and also of what would
>>> seem
>> acceptable).
>> 
>> The time doesn't include the cost of handling of interrupt. We just
>> record it at the entry of interrupt handler. So the cost should bigger
>> than 4.3 us if taking the handing cost into consideration. And the
>> costs will much bigger if there are more pass-through VMs runs. We can
>> start from ATS case firstly. And apply it to non-ATS case later if the
>> async approach doesn't hurt the performance.
> 
> In which case we're back to the question I raised originally: How do
> you explain the time difference if the interrupt delivery overhead isn't 
> included?

which one? 1.4us for sync case and 4.3us for async case?
> 
> Jan


Best regards,
Yang



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel