Re: [Xen-devel] FW: VT-d async invalidation for Device-TLB.

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 16.06.15 at 09:59, <yang.z.zhang@xxxxxxxxx> wrote:
> Jan Beulich wrote on 2015-06-16:
>>>>> On 16.06.15 at 05:07, <yang.z.zhang@xxxxxxxxx> wrote:
>>> Jan Beulich wrote on 2015-06-15:
>>>>>>> On 13.06.15 at 16:44, <quan.xu@xxxxxxxxx> wrote:
>>>>>> On 12.06.15 at 14:47, <JBeulich@xxxxxxxx> wrote:
>>>>>>>>> On 12.06.15 at 04:40, <quan.xu@xxxxxxxxx> wrote:
>>>>>>> I tested it by a Myri-10G Dual-Protocol NIC, which is an ATS device.
>>>>>>> for an invalidation:
>>>>>>>  By sync way, it takes about 1.4 ms.
>>>>>>>  By async way, it takes about 4.3 ms.
>>> 
>>> It is a typo for the time: should be 1.4 us and 4.3 us.
>>> 
>>>>>> 
>>>>>> What's the theory on why this is? After all, it shouldn't matter
>>>>>> how the completion of the invalidation gets signaled.
>>>>>> 
>>>>> 
>>>>> Let me introduce more about how I get these test data.
>>>>> 
>>>>> For sync way, I get the time by NOW() macro, when I update Queue Tail
>>>>> Register  (qinval_update_qtail()). Then get time by NOW() macro in
>>>>> current spinning when it has wrote the value in the Status Data field
>>>>> to the address specified in the Status Address. The difference of
>>>>> these 2 value is the time of an sync invalidation.
>>>>> 
>>>>> 
>>>>> For async way, as the previous email, I have introduced the IF bit
>>>>> of Invalidation Wait Descriptor.
>>>>> (IF: Indicate the invalidation wait descriptor completion by
>>>>> generating an invalidation completion event per the programing of
>>>>> the Invalidation Completion Event Registers.) I have implemented
>>>>> an interrupt for invalidation completion event.
>>>>> Also I get the time by NOW() macro, when I update Queue Tail
>>>>> Register (by qinval_update_qtail()).
>>>>> Then get time by NOW() macro in invalidation completion event handler.
>>>>> The difference of these 2 value is the time of an async invalidation.
>>>> 
>>>> Okay, thanks for the explanation. As this includes the time it
>>>> takes to deliver and (partly) handle the interrupt, the difference
>>>> is of course within what one would expect (and also of what would
>>>> seem
>>> acceptable).
>>> 
>>> The time doesn't include the cost of handling of interrupt. We just
>>> record it at the entry of interrupt handler. So the cost should bigger
>>> than 4.3 us if taking the handing cost into consideration. And the
>>> costs will much bigger if there are more pass-through VMs runs. We can
>>> start from ATS case firstly. And apply it to non-ATS case later if the
>>> async approach doesn't hurt the performance.
>> 
>> In which case we're back to the question I raised originally: How do
>> you explain the time difference if the interrupt delivery overhead isn't 
> included?
> 
> which one? 1.4us for sync case and 4.3us for async case?

The difference between the two (i.e. why is the async variant
three times as long).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel