Re: [Xen-devel] Dom0 hypercall for adding and removing PCI devices

[Keir Fraser <keir.fraser@xxxxxxxxxxxxx>]

Then what you do already (leave the device assigned to dom0) is the best you
can do. That's easy.

 -- Keir

On 24/7/08 10:14, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:

> VT-d spec says: Software must not modify fields other than the Present (P)
> field of currently present root-entries or context-entries. If modifications
> to other fields are required, software must first make these entries
> not-present (P=0), which requires serial invalidation of context-cache and
> IOTLB, and then transition them to present (P=1) state along with the
> modifications.
> 
> So your suggestion is not feasible.
> 
> Randy (Weidong)
> 
> Keir Fraser wrote:
>> Exactly my thought.
>> 
>>  K.
>> 
>> On 24/7/08 09:43, "Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:
>> 
>>> Isn't enough to first switch VT-d page table, and then flush IOTLB?
>>> As long as RMRRs are kept same in two VT-d tables, and in any
>>> time a valid entry (either in IOTLB or by walking) can be found,
>>> above sequence seems complete.
>>> 
>>> Thanks,
>>> Kevin
>>> 
>>>> From: Keir Fraser
>>>> Sent: 2008年7月24日 16:38
>>>> 
>>>> Can the pagetable switch not be made atomic (i.e., so that the RMRR
>>>> regions appear continuously available throughout)? I'd have thought
>>>> that would just naturally happen.
>>>> 
>>>> If creating/destroying RMRR mappings is part of the switch
>>>> operation, you'd have to destroy RMRR mappings in the dom0 table
>>>> after the switch, and create RMRR mappings in the fallback table
>>>> before the switch. Or just have RMRR mappings always mapped into
>>>> all tables. 
>>>> 
>>>> -- Keir
>>>> 
>>>> On 24/7/08 09:32, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>> 
>>>>> We found USB (has RMRR) is firstly assigned to dom0, but
>>>>> pci_bus_probe() failed, then it was removed from dom0. The
>>>>> removing needs switch to RMRR VT-d page table. At the same time,
>>>>> BIOS was using its RMRR.
>>>>> 
>>>>> Randy (Weidong)
>>>>> 
>>>>> Keir Fraser wrote:
>>>>>> If a device is assigned to a domain (in this case dom0) then that
>>>>>> domain's VT-d pagetables will contain the RMRR mappings for that
>>>>>> device. Hence BIOS can perform DMA in those RMRR-indicated regions
>>>>>> without swapping to and fro between domain tables and fallback
>>>>>> RMRR tables. The new fallback RMRR table would be just that -- a
>>>>>> fallback table used for any device not currently assigned to any
>>>>>> domain (and hence those devices should only have DMAs initiated
>>>>>> by the BIOS, if at all).
>>>>>> 
>>>>>>  -- Keir
>>>>>> 
>>>>>> On 24/7/08 09:20, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>>>> 
>>>>>>> I have another concern, when BIOS is initiating DMA operation
>>>>>>> using RMRR, can we use RMRR VT-d page table to replace dom0 VT-d
>>>>>>> page table? Does it result in some DMA failures?
>>>>>>> 
>>>>>>> Randy (weidong)
>>>>>>> 
>>>>>>> Han, Weidong wrote:
>>>>>>>> Espen Skoglund wrote:
>>>>>>>>> [Keir Fraser]
>>>>>>>>>> On 23/7/08 10:26, "Han, Weidong" <weidong.han@xxxxxxxxx>
>>>>>>>>>> wrote: 
>>>>>>>>>>>> So this would be one extra VT-d pagetable, for the whole
>>>>>>>>>>>> system, which would be the fallback location for RMRR
>>>>>>>>>>>> mappings for devices which are currently not assigned to
>>>>>>>>>>>> any domain? Thus allowing firmware to successfully initiate
>>>>>>>>>>>> DMA operations on those devices? Sounds sensible.
>>>>>>>>> 
>>>>>>>>> Well, the VT-d page table for RMRR devices need not contain the
>>>>>>>>> whole system memory---only those regions specified in the DMAR
>>>>>>>>> tables. 
>>>>>>>>> 
>>>>>>>>>>> Is it possible that idle_domain owns the RMRR VT-d page
>>>>>>>>>>> table? 
>>>>>>>>> 
>>>>>>>>>> If that's a convenient place to stash it then why not? Either
>>>>>>>>>> way, seems you're going to have it special-cased in the code
>>>>>>>>>> as fallback owner for unassigned devices. It's possible that
>>>>>>>>>> having it stashed in the idle domain will simply make the
>>>>>>>>>> code more confusing. I'm not sure though.
>>>>>>>>> 
>>>>>>>>> Right.  I don't see any particular good reason to associate it
>>>>>>>>> with the idle domain.  As noted above, the page table need not
>>>>>>>>> even cover the whole memory, and it will never change after
>>>>>>>>> being set up at boot time.  If special case code is needed
>>>>>>>>> anyway, then one might as well install a custom VT-d page
>>>>>>>>> table. 
>>>>>>>> 
>>>>>>>> What does "custom VT-d page table" mean?
>>>>>>>> 
>>>>>>>> Due to domain id is not the same with DID field in context, we
>>>>>>>> can reverse a DID for RMRR VT-d page table, it can avoid to
>>>>>>>> associate with idle domain.
>>>>>>>> 
>>>>>>>> Currently we reassign the device from dom0 to target domain when
>>>>>>>> assign a device, and return the device to dom0 when target
>>>>>>>> domain tears down. It's not correct due to some devices may be
>>>>>>>> not assigned to any domain. Current device_assigned() also
>>>>>>>> needs to be changed. Maybe it needs more changes on VT-d.
>>>>>>>> 
>>>>>>>> I have some concerns, maybe I missed something. Why did you use
>>>>>>>> dom0 hypercall approach to replace original method? What's the
>>>>>>>> main benefit? I also wonder it's suitable to wrap
>>>>>>>> pci_bus_probe() function.
>>>>>>>> 
>>>>>>>> Randy (Weidong)
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> If supported by hardware, the extra page tables can even be
>>>>>>>>> skipped altogether and the device marked as having passthrough
>>>>>>>>> access. That would give the RMRR device complete access to
>>>>>>>>> system memory, though.
>>>>>>>>> 
>>>>>>>>> eSk
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-devel
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel