Re: [Xen-devel] [qemu-upstream-4.11-testing test] 136184: regressions - FAIL

[Julien Grall <julien.grall@xxxxxxx>]

Hi,

Answering to myself.

On 5/17/19 8:00 PM, Julien Grall wrote:
> Hi,
>
> On 5/17/19 6:23 PM, Anthony PERARD wrote:
> > On Thu, May 16, 2019 at 10:38:54PM +0100, Julien Grall wrote:
> > > Hi Anthony,
> > >
> > > Thank you for CCing me.
> > >
> > > On 5/16/19 11:37 AM, Anthony PERARD wrote:
> > > > On Wed, May 15, 2019 at 07:48:17PM +0000, osstest service owner wrote:
> > > > > flight 136184 qemu-upstream-4.11-testing real [real]
> > > > > http://logs.test-lab.xenproject.org/osstest/logs/136184/
> > > > >
> > > > > Regressions :-(
> > > > >
> > > > > Tests which did not succeed and are blocking,
> > > > > including tests which could not be run:
> > > > >    build-arm64-pvops               <job status>                 
> > > > > broken  in 134594
> > > > >    build-arm64                     <job status>                 
> > > > > broken  in 134594
> > > > >    build-arm64-xsm                 <job status>                 
> > > > > broken  in 134594
> > > > >    build-arm64-xsm            4 host-install(4) broken in 134594 
> > > > > REGR. vs. 125575
> > > > >    build-arm64-pvops          4 host-install(4) broken in 134594 
> > > > > REGR. vs. 125575
> > > > >    build-arm64                4 host-install(4) broken in 134594 
> > > > > REGR. vs. 125575
> > > > >    test-arm64-arm64-libvirt-xsm  7 xen-boot                 fail 
> > > > > REGR. vs. 125575
> > > > >    test-arm64-arm64-xl           7 xen-boot                 fail 
> > > > > REGR. vs. 125575
> > > > >    test-arm64-arm64-xl-xsm       7 xen-boot                 fail 
> > > > > REGR. vs. 125575
> > > > >    test-arm64-arm64-xl-credit2   7 xen-boot                 fail 
> > > > > REGR. vs. 125575
> > > > Ian, Julien,
> > > >
> > > > I can't figure out why Xen consistently fails to boot on rochester* in
> > > > the qemu-upstream-4.11-testing flights. The xen-4.11-testing seems to
> > > > pass.
> > > >
> > > > At boot, the boot loader seems to load blobs, but when it's time to Xen
> > > > to shine, there are no output from Xen on the serial.
> > > The serial console is initializing fairly late in the process. Any 
> > > useful
> > > message (such as memory setup or even part of the interrupts) will be 
> > > hide
> > > out. For getting them, you need earlyprintk. Unfortunately they can't be
> > > configured at runtime today :(.
> > I think I managed to run the job with earlyprintk on rochester, but
> > Xen have booted:
> > http://logs.test-lab.xenproject.org/osstest/logs/136451/
> Yes this is with earlyprintk. That's going to be fun to reproduce if 
> earlyprintk modifies the behavior.
> I think we can interpret as earlyprintk add enough latency to make 
> everything working.
> There are two possible issues I can think of:
>      1) The boot code does not follow the Arm Arm, so it may be possible 
> the board is doing something different compare to the other regarding 
> the memory. IIRC, this is the first hardware we have with core not 
> directly designed by Arm.
>      2) We are missing some errata in Xen. Linux contains 6 errata for 
> that platform. Looking at them, I don't think they matter for boot time.
> 1) is currently been looked at (see MM-PART* patches on the ML). 2) 
> should probably be addressed at some point, but I may not be able to 
> send them as Arm employee (we tend to avoid sending patch showing 
> brokenness in partner silicon).
Ian kindly started a couple of jobs over the week-end to confirm whether 
it can be reproduced on laxton* (Seattle board).
The same error cannot be reproduced on laxton*. Looking at the test 
history, it looks like qemu-upstream-4.12-testing flight has run 
successfully a few times on rochester*. So we may have fixed the error 
in Xen 4.12.
Potential candidates would be:
   - 00c96d7742 "xen/arm: mm: Set-up page permission for Xen mappings 
earlier on"
   - f60658c6ae "xen/arm: Stop relocating Xen"

Ian, is it something the bisector could automatically look at?
If not, I will need to find some time and borrow the board to bisect the 
issues.
Cheers,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel