|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] Xen Dom0 boot failure on platform that supports ARM GICv4
Hi Julien, Thanks for taking a look at this. > -----Original Message----- > From: Julien Grall [mailto:julien.grall@xxxxxxx] > Sent: 03 September 2018 17:13 > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx>; > xen-devel@xxxxxxxxxxxxx > Cc: sstabellini@xxxxxxxxxx; Linuxarm <linuxarm@xxxxxxxxxx>; Andre > Przywara <andre.przywara@xxxxxxx> > Subject: Re: Xen Dom0 boot failure on platform that supports ARM GICv4 > > > > On 03/09/18 15:53, Shameerali Kolothum Thodi wrote: > > Hi, > > Hello, > > > I am trying to boot xen(stable-4.11) on one of our ARM64 boards which > > has support for GICv4. > > > > But dom0(kernel 4.18) boot fails with the below trap, > > > > XEN) ............done. > > (XEN) Std. Loglevel: All > > (XEN) Guest Loglevel: All > > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch > > input to Xen) > > (XEN) Freed 304kB init memory. > > (XEN) traps.c:2007:d0v0 HSR=0x93800004 pc=0xffff00000841af04 > > gva=0xffff00000b10ffe8 gpa=0x004000aa10ffe8 > > Which bits of Linux is trying to access the region? I think it is the gic_iterate_rdists() as the offset just before this is ffe8, which is GICR_PIDR2 > > > > After a bit of debugging, it looks like, the GICR size used in > vgic_v3_domain_init() > > is GICv4 GICR size(256K) and this upsets the first_cpu calculations. > > Can you expand what you mean by upset? What's wrong with the first_cpu > calculations. What I meant is, since this is a GICv4, the vgic_v3_hw.regions[i]->size is set to 256K and since first_cpu is calculated like, first_cpu += size /GICV3_GICR_SIZE; gets wrong as what I am seeing is, (XEN) frst_cpu 2 (XEN) first_cpu 4 (XEN) first_cpu 6 (XEN) first_cpu 8 (XEN) first_cpu 10 (XEN) first_cpu 12 (XEN) first_cpu 14 ..... (XEN) first_cpu 192 But the original number of CPUS are only 96. Hence I thought this is wrong. > > > > Since dom0 gicv3 is also an emulated one, I think the size should be > > restricted to use the GICv3 GICR size(128K). I have made the below > > changes and is able to boot dom0 now. > > > > But not sure, this is the right approach to fix the issue. Please let me > > know your thoughts. > > > > Thanks, > > Shameer > > > > ---->8------------- > > > > diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c > > index b2ed0f8..bf028cc 100644 > > --- a/xen/arch/arm/gic-v3.c > > +++ b/xen/arch/arm/gic-v3.c > > @@ -1783,7 +1783,8 @@ static int __init gicv3_init(void) > > reg = readl_relaxed(GICD + GICD_TYPER); > > intid_bits = GICD_TYPE_ID_BITS(reg); > > > > - vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions, > intid_bits); > > + vgic_v3_setup_hw(dbase, gicv3.rdist_count, gicv3.rdist_regions, > > + intid_bits, gic_dist_supports_dvis()); > > gicv3_init_v2(); > > > > spin_lock_init(&gicv3.lock); > > diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c > > index 4b42739..0f53d88 100644 > > --- a/xen/arch/arm/vgic-v3.c > > +++ b/xen/arch/arm/vgic-v3.c > > @@ -59,18 +59,21 @@ static struct { > > unsigned int nr_rdist_regions; > > const struct rdist_region *regions; > > unsigned int intid_bits; /* Number of interrupt ID bits */ > > + bool dvis; > > } vgic_v3_hw; > > > > void vgic_v3_setup_hw(paddr_t dbase, > > unsigned int nr_rdist_regions, > > const struct rdist_region *regions, > > - unsigned int intid_bits) > > + unsigned int intid_bits, > > + bool dvis) > > { > > vgic_v3_hw.enabled = true; > > vgic_v3_hw.dbase = dbase; > > vgic_v3_hw.nr_rdist_regions = nr_rdist_regions; > > vgic_v3_hw.regions = regions; > > vgic_v3_hw.intid_bits = intid_bits; > > + vgic_v3_hw.dvis = dvis; > > } > > > > static struct vcpu *vgic_v3_irouter_to_vcpu(struct domain *d, uint64_t > irouter) > > @@ -1673,6 +1676,9 @@ static int vgic_v3_domain_init(struct domain *d) > > { > > paddr_t size = vgic_v3_hw.regions[i].size; > > > > + if (vgic_v3_hw.dvis && (size == GICV4_GICR_SIZE)) > > + size = GICV3_GICR_SIZE; > > vgic_v3_hw.regions is describing the regions in the layout that could > hold re-distributor. You can have multiple re-distributor per region. > > The variable size holds the size of the region, not the size of the > re-distributor. > > I am not sure to understand why you want to restrict the size of the > region here because GICV4_GICR_SIZE is a multiple of GICV3_GICR_SIZE. So > you should be able to fit 2 re-distributors per region. > > It looks like to me the re-distributor regions are not reported > correctly or Dom0 thinks it is on GICv4. Can you provide a bit more > details on the function that cause the crash and some logs from Linux? Ok. I added few prints along the vgic mmio read path and this is what happens before the trap. vgic_v3_rdistr_mmio_read() get_vcpu_from_rdist() -->returns NULL here for 0x004000aa10ffe8 which actually belongs to cpu id 48 as per the log below (XEN) 96 CPUs enabled, 96 CPUs total (XEN) SMP: Allowing 96 CPUs (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 100000 KHz (XEN) GICv3 initialization: (XEN) gic_dist_addr=0x000000aa000000 (XEN) gic_maintenance_irq=25 (XEN) gic_rdist_stride=0 (XEN) gic_rdist_regions=96 (XEN) redistributor regions: (XEN) - region 0: 0x000000aa100000 - 0x000000aa140000 (XEN) - region 1: 0x000000aa140000 - 0x000000aa180000 (XEN) - region 2: 0x000000aa180000 - 0x000000aa1c0000 (XEN) - region 3: 0x000000aa1c0000 - 0x000000aa200000 (XEN) - region 4: 0x000000aa200000 - 0x000000aa240000 (XEN) - region 5: 0x000000aa240000 - 0x000000aa280000 (XEN) - region 6: 0x000000aa280000 - 0x000000aa2c0000 (XEN) - region 7: 0x000000aa2c0000 - 0x000000aa300000 (XEN) - region 8: 0x000000aa300000 - 0x000000aa340000 (XEN) - region 9: 0x000000aa340000 - 0x000000aa380000 (XEN) - region 10: 0x000000aa380000 - 0x000000aa3c0000 (XEN) - region 11: 0x000000aa3c0000 - 0x000000aa400000 (XEN) - region 12: 0x000000aa400000 - 0x000000aa440000 (XEN) - region 13: 0x000000aa440000 - 0x000000aa480000 (XEN) - region 14: 0x000000aa480000 - 0x000000aa4c0000 (XEN) - region 15: 0x000000aa4c0000 - 0x000000aa500000 (XEN) - region 16: 0x000000aa500000 - 0x000000aa540000 (XEN) - region 17: 0x000000aa540000 - 0x000000aa580000 (XEN) - region 18: 0x000000aa580000 - 0x000000aa5c0000 (XEN) - region 19: 0x000000aa5c0000 - 0x000000aa600000 (XEN) - region 20: 0x000000aa600000 - 0x000000aa640000 (XEN) - region 21: 0x000000aa640000 - 0x000000aa680000 (XEN) - region 22: 0x000000aa680000 - 0x000000aa6c0000 (XEN) - region 23: 0x000000aa6c0000 - 0x000000aa700000 (XEN) - region 24: 0x000000ae100000 - 0x000000ae140000 (XEN) - region 25: 0x000000ae140000 - 0x000000ae180000 (XEN) - region 26: 0x000000ae180000 - 0x000000ae1c0000 (XEN) - region 27: 0x000000ae1c0000 - 0x000000ae200000 (XEN) - region 28: 0x000000ae200000 - 0x000000ae240000 (XEN) - region 29: 0x000000ae240000 - 0x000000ae280000 (XEN) - region 30: 0x000000ae280000 - 0x000000ae2c0000 (XEN) - region 31: 0x000000ae2c0000 - 0x000000ae300000 (XEN) - region 32: 0x000000ae300000 - 0x000000ae340000 (XEN) - region 33: 0x000000ae340000 - 0x000000ae380000 (XEN) - region 34: 0x000000ae380000 - 0x000000ae3c0000 (XEN) - region 35: 0x000000ae3c0000 - 0x000000ae400000 (XEN) - region 36: 0x000000ae400000 - 0x000000ae440000 (XEN) - region 37: 0x000000ae440000 - 0x000000ae480000 (XEN) - region 38: 0x000000ae480000 - 0x000000ae4c0000 (XEN) - region 39: 0x000000ae4c0000 - 0x000000ae500000 (XEN) - region 40: 0x000000ae500000 - 0x000000ae540000 (XEN) - region 41: 0x000000ae540000 - 0x000000ae580000 (XEN) - region 42: 0x000000ae580000 - 0x000000ae5c0000 (XEN) - region 43: 0x000000ae5c0000 - 0x000000ae600000 (XEN) - region 44: 0x000000ae600000 - 0x000000ae640000 (XEN) - region 45: 0x000000ae640000 - 0x000000ae680000 (XEN) - region 46: 0x000000ae680000 - 0x000000ae6c0000 (XEN) - region 47: 0x000000ae6c0000 - 0x000000ae700000 (XEN) - region 48: 0x004000aa100000 - 0x004000aa140000 (XEN) - region 49: 0x004000aa140000 - 0x004000aa180000 (XEN) - region 50: 0x004000aa180000 - 0x004000aa1c0000 (XEN) - region 51: 0x004000aa1c0000 - 0x004000aa200000 (XEN) - region 52: 0x004000aa200000 - 0x004000aa240000 (XEN) - region 53: 0x004000aa240000 - 0x004000aa280000 (XEN) - region 54: 0x004000aa280000 - 0x004000aa2c0000 (XEN) - region 55: 0x004000aa2c0000 - 0x004000aa300000 (XEN) - region 56: 0x004000aa300000 - 0x004000aa340000 (XEN) - region 57: 0x004000aa340000 - 0x004000aa380000 (XEN) - region 58: 0x004000aa380000 - 0x004000aa3c0000 (XEN) - region 59: 0x004000aa3c0000 - 0x004000aa400000 (XEN) - region 60: 0x004000aa400000 - 0x004000aa440000 (XEN) - region 61: 0x004000aa440000 - 0x004000aa480000 (XEN) - region 62: 0x004000aa480000 - 0x004000aa4c0000 (XEN) - region 63: 0x004000aa4c0000 - 0x004000aa500000 (XEN) - region 64: 0x004000aa500000 - 0x004000aa540000 (XEN) - region 65: 0x004000aa540000 - 0x004000aa580000 (XEN) - region 66: 0x004000aa580000 - 0x004000aa5c0000 (XEN) - region 67: 0x004000aa5c0000 - 0x004000aa600000 (XEN) - region 68: 0x004000aa600000 - 0x004000aa640000 (XEN) - region 69: 0x004000aa640000 - 0x004000aa680000 (XEN) - region 70: 0x004000aa680000 - 0x004000aa6c0000 (XEN) - region 71: 0x004000aa6c0000 - 0x004000aa700000 (XEN) - region 72: 0x004000ae100000 - 0x004000ae140000 (XEN) - region 73: 0x004000ae140000 - 0x004000ae180000 (XEN) - region 74: 0x004000ae180000 - 0x004000ae1c0000 (XEN) - region 75: 0x004000ae1c0000 - 0x004000ae200000 (XEN) - region 76: 0x004000ae200000 - 0x004000ae240000 (XEN) - region 77: 0x004000ae240000 - 0x004000ae280000 (XEN) - region 78: 0x004000ae280000 - 0x004000ae2c0000 (XEN) - region 79: 0x004000ae2c0000 - 0x004000ae300000 (XEN) - region 80: 0x004000ae300000 - 0x004000ae340000 (XEN) - region 81: 0x004000ae340000 - 0x004000ae380000 (XEN) - region 82: 0x004000ae380000 - 0x004000ae3c0000 (XEN) - region 83: 0x004000ae3c0000 - 0x004000ae400000 (XEN) - region 84: 0x004000ae400000 - 0x004000ae440000 (XEN) - region 85: 0x004000ae440000 - 0x004000ae480000 (XEN) - region 86: 0x004000ae480000 - 0x004000ae4c0000 (XEN) - region 87: 0x004000ae4c0000 - 0x004000ae500000 (XEN) - region 88: 0x004000ae500000 - 0x004000ae540000 (XEN) - region 89: 0x004000ae540000 - 0x004000ae580000 (XEN) - region 90: 0x004000ae580000 - 0x004000ae5c0000 (XEN) - region 91: 0x004000ae5c0000 - 0x004000ae600000 (XEN) - region 92: 0x004000ae600000 - 0x004000ae640000 (XEN) - region 93: 0x004000ae640000 - 0x004000ae680000 (XEN) - region 94: 0x004000ae680000 - 0x004000ae6c0000 (XEN) - region 95: 0x004000ae6c0000 - 0x004000ae700000 (XEN) GICv3: using at most 57344 LPIs on the host. (XEN) GICv3: 672 lines, (IID 00030736). (XEN) GICv3: Found ITS @0x202100000 (XEN) GICv3: CPU0: Found redistributor in region 0 @0000000040037000 (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Defaulting to alternative key handling; send 'A' to switch to normal mode. (XEN) Allocated console ring of 1024 KiB. (XEN) Bringing up CPU1 ....... If I remember correctly there was no logs from Dom0, but I need to double check the Dom0 cmdline option to see earlycon was set. I could also enable/add any prints that you think will help and rerun. Please let me know. > Also, which Linux version are you using? 4.18-rc1. Thanks, Shameer > Cheers, > > -- > Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxx https://lists.xenproject.org/mailman/listinfo/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |