Re: [Xen-devel] Xen-unstable PVHdom0: Assertion 'IS_ALIGNED(dfn_x(dfn), (1ul << page_order))' failed at iommu.c:324

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

On 24/01/2019 11:11, Roger Pau Monné wrote:
> On Thu, Jan 24, 2019 at 10:25:33AM +0100, Sander Eikelenboom wrote:
>> On 24/01/2019 08:50, Roger Pau Monné wrote:
>>> On Wed, Jan 23, 2019 at 08:56:48PM +0100, Sander Eikelenboom wrote:
>>>> On 23/01/2019 19:25, Roger Pau Monné wrote:
>>>>> On Wed, Jan 23, 2019 at 12:39:21AM +0100, Sander Eikelenboom wrote:
>>>>>> On 22/01/2019 17:14, Roger Pau Monné wrote:
>>>>>>> On Sun, Jan 20, 2019 at 11:09:25PM +0100, Sander Eikelenboom wrote:
>>>>>>>> On 18/01/2019 18:56, Roger Pau Monné wrote:
>>>>>>>>> On Fri, Jan 18, 2019 at 03:17:57PM +0100, Sander Eikelenboom wrote:
>>>>>>>>>> On 18/01/2019 13:50, Roger Pau Monné wrote:
>>>>>>>>>>> On Fri, Jan 18, 2019 at 01:03:04PM +0100, Sander Eikelenboom wrote:
>>>>>>>>>>>> Hi Roger,
>>>>>>>>>>>>
>>>>>>>>>>>> I gave PVH dom0 a spin, see how far I would get.
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>> With current xen-unstable unfortunately not that far, i got the 
>>>>>>>>>>>> splat below.
>>>>>>>>>>>
>>>>>>>>>>> Yes, this was already reported:
>>>>>>>>>>>
>>>>>>>>>>> https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg01030.html
>>>>>>>>>>>> If you need more info, would like me to test a patch (or some 
>>>>>>>>>>>> other git tree/branch), 
>>>>>>>>>>>> I will be happy to give it a spin !
>>>>>>>>>>>
>>>>>>>>>>> Paul is working on a fix, but in the meantime just removing the
>>>>>>>>>>> assertions should be fine:
>>>>>>>>>>>
>>>>>>>>>>> ---8<---
>>>>>>>>>>> diff --git a/xen/drivers/passthrough/iommu.c 
>>>>>>>>>>> b/xen/drivers/passthrough/iommu.c
>>>>>>>>>>> index bd1af35a13..98e6fc35e2 100644
>>>>>>>>>>> --- a/xen/drivers/passthrough/iommu.c
>>>>>>>>>>> +++ b/xen/drivers/passthrough/iommu.c
>>>>>>>>>>> @@ -321,9 +321,6 @@ int iommu_map(struct domain *d, dfn_t dfn, 
>>>>>>>>>>> mfn_t mfn,
>>>>>>>>>>>      if ( !iommu_enabled || !hd->platform_ops )
>>>>>>>>>>>          return 0;
>>>>>>>>>>>  
>>>>>>>>>>> -    ASSERT(IS_ALIGNED(dfn_x(dfn), (1ul << page_order)));
>>>>>>>>>>> -    ASSERT(IS_ALIGNED(mfn_x(mfn), (1ul << page_order)));
>>>>>>>>>>> -
>>>>>>>>>>>      for ( i = 0; i < (1ul << page_order); i++ )
>>>>>>>>>>>      {
>>>>>>>>>>>          rc = hd->platform_ops->map_page(d, dfn_add(dfn, i), 
>>>>>>>>>>> mfn_add(mfn, i),
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I gave that a spin and i now get a seemingly endless stream of 
>>>>>>>>>> IO_PAGE_FAULTs
>>>>>>>>>
>>>>>>>>> You shouldn't get those page faults since they are for addresses that
>>>>>>>>> belong to a reserved region, and that should be mapped into the p2m.
>>>>>>>>> I've just tested on my AMD box and I'm also seeing errors (albeit
>>>>>>>>> different ones), so I guess something broke since I last fixed PVH
>>>>>>>>> Dom0 to boot on AMD hardware.
>>>>>>>>>
>>>>>>>>> I've also tested commit:
>>>>>>>>>
>>>>>>>>> commit fad6ba64a8c98bebb9374f390cc255fac05237ab (HEAD)
>>>>>>>>> Author: Roger Pau Monné <roger.pau@xxxxxxxxxx>
>>>>>>>>> Date:   Fri Nov 30 12:10:00 2018 +0100
>>>>>>>>> amd/iommu: skip host bridge devices when updating IOMMU page tables
>>>>>>>>>
>>>>>>>>> And it works on my AMD box and I'm able to boot as a PVH Dom0. Can you
>>>>>>>>> give this commit a spin?
>>>>>>>>>
>>>>>>>>> Thanks, Roger.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Roger,
>>>>>>>>
>>>>>>>> Tested that commit, but that didn't help.
>>>>>>>
>>>>>>> Thanks! Sorry for the delay, I got sidetracked with something else.
>>>>>>
>>>>>> No problem, it's not too urgent and probably a busy time with the 
>>>>>> remaining 4.12 stuff.
>>>>>>  
>>>>>>> Can you please post the serial log when using the above commit?
>>>>>>
>>>>>> Sure, I attached a log of:
>>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PVH unsuccesful boot
>>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PV    succesful boot
>>>>>
>>>>> Thanks. So you get the same IO page faults.
>>>>>
>>>>> I don't seem to be able to reproduce this behaviour on my AMD box, but
>>>>> that might be just luck. I've been finding some issues today related
>>>>> to the IOMMU, could you give the following patch a spin and paste the
>>>>> serial log that you get.
>>>>
>>>> Hi Roger,
>>>>
>>>> Sure, on top of what ?
>>>> - fad6ba64a8c98bebb9374f390cc255fac05237ab ?
>>>> - xen-unstable ?
>>>> - xen-unstable + Paul's patch ?
>>>
>>> Hello,
>>>
>>> Sorry for not proving the right context, let's try on top of
>>> xen-unstable + Paul's patch.
>>>
>>> Thanks, Roger.
>>>
>>
>> Seems to be giving the same result (stream of IO_PAGE_FAULTs), serial
>> log attached.
> 
> Thanks, I think I've figured out what's wrong. I've prepared a git
> branch for you to test:
> 
> git://xenbits.xen.org/people/royger/xen.git iommu-fixes
> 
> Could you give this a try?
> 
> Thanks, Roger.

Hi Roger,

The good news is, with that branch, dom0 boots on both PVH and PV !
The bad news is, I can't boot up a PVH guest (but perhaps this warrants a 
separate mail thread / change of subject):

xc: error: panic: xc_dom_boot.c:159: xc_dom_boot_domU_map: failed to mmap domU 
pages 0x1000+0x2426 [mmap, errno=22 (Invalid argument)]: Internal error
libxl: error: libxl_dom.c:760:libxl__build_dom: xc_dom_build_image failed: 
Invalid argument
libxl: error: libxl_create.c:1286:domcreate_rebuild_done: Domain 1:cannot 
(re-)build domain: -3
libxl: error: libxl_domain.c:1038:libxl__destroy_domid: Domain 1:Non-existant 
domain
libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 1:Unable to 
destroy guest
libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 1:Destruction of 
domain failed

There is more in the attached serial.log

Thanks so far !

--
Sander

Attachment: serial.log
Description: Text Data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel