Re: [Xen-devel] pvops: AHCI problems with SB600

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Wed, Sep 23, 2009 at 01:30:44PM -0700, Jeremy Fitzhardinge wrote:
> On 09/23/09 13:09, Jeremy Fitzhardinge wrote:
> >             if (!no_iommu &&
> >                 max_pfn > MAX_DMA32_PFN &&
> >                 !printed_gart_size_msg) {
> >                     printk(KERN_ERR "you are using iommu with agp, but GART 
> > size is less than 64M\n");
> >                     printk(KERN_ERR "please increase GART size in your BIOS 
> > setup\n");
> >                     printk(KERN_ERR "if BIOS doesn't have that option, 
> > contact your HW vendor!\n");
> >                     printed_gart_size_msg = 1;
> >             }
> >   
> 
> Oh, that's the wrong error message, but the other one has similar
> predicates.  Hm, but it also skips the test if (swiotlb && !valid_agp)...

Right. We don't set the swiotlb. The reason being if you do set it then
the original SWIOTLB kicks in.

The weird part is that the function you copied-n-pasted (gart_iommu_hole_init)
only detectes and allocates a buffer. It does not set the dma_ops at all.
Setting of the dma_ops is done via the gart_iommu_init() call which is done
much later. But with Xen-SWIOTLB already initialized, the gart_iommu_init()
quits right away.

So the kernel sets the dma_ops to the Xen SWIOTLB, and it
allocates an extra 64MB chunk of memory for the GART, which is not
used, and ... somehow all of the ioremap_nocache functions stop working
correctly. Maybe the ioremap_nocache does use some of that memory that
the gart_iommu_hole_init allocated?

With this patch, the GART is forcefully disabled, and the kernel boots fine
(with 6GB, 8GB, etc).

diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 6b76948..1101a9f 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -122,6 +122,8 @@ void __init pci_iommu_alloc(void)
         * The order of these functions is important for
         * fall-back/fail-over reasons
         */
+       xen_swiotlb_init();
+
        gart_iommu_hole_init();
 
        detect_calgary();
@@ -130,8 +132,6 @@ void __init pci_iommu_alloc(void)
 
        amd_iommu_detect();
 
-       xen_swiotlb_init();
-
        pci_swiotlb_init();
 }
 
diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
index 5e2c856..00f2260 100644
--- a/arch/x86/xen/pci-swiotlb.c
+++ b/arch/x86/xen/pci-swiotlb.c
@@ -43,6 +43,10 @@
 #include <xen/page.h>
 #include <xen/xen-ops.h>
 
+
+#include <linux/pci.h>
+#include <asm/gart.h>
+
 #define OFFSET(val,align) ((unsigned long)     \
                           ( (val) & ( (align) - 1)))
 
@@ -985,5 +989,9 @@ void __init xen_swiotlb_init(void)
                xen_swiotlb_init_with_default_size(64 * (1<<20));       /* 
default to 64MB */
                dma_ops = &xen_swiotlb_dma_ops;
                iommu_detected = 1;
+#ifdef CONFIG_GART_IOMMU
+               gart_iommu_aperture_disabled = 1;
+#endif
+
        }
 }


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel