Re: [Xen-devel] Vt-d not working with 3.4.1

Sigh...

Looks like I have to compile the nouveau driver sources against the pvops 
kernel instead and give up official nvidia drivers. Any ideas it might work?

Because for pvops dom 0 kernel, CONFIG_DRM_NOUVEAU is not supported.
CONFIG_DRM_NOUVEAU is only supported in the default shipping Fedora 11 kernel.

When I look at this wiki page http://nouveau.freedesktop.org/wiki/InstallDRM , 
I got a bit confused as there are too many alternatives suggested.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering) 
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
Alma Maters: Singapore Polytechnic, National University of Singapore



On 08/21/2009 02:03 AM, Andrew Lyon wrote:

On Thu, Aug 20, 2009 at 5:41 PM, Mr. Teo En Ming (Zhang
Enming)<enming.teo@xxxxxxxxxxxxxxx>  wrote:

Dear All,

I followed the XEN - NVIDIA - STEP BY STEP instructions at nV news forums
but I still couldn't get X to work under Xen pv-ops domain 0 kernel. It
still shows me a freezed blank screen. I couldn't switch to any tty (1 to 7
inclusive).

Here are the scenarios I have tried:

1)     IGNORE_XEN_PRESENCE=y ./NVIDIA-Linux-x86_64-185.18.31-pkg2.run

2)     IGNORE_XEN_PRESENCE=y CC="gcc -DNV_VMAP_4_PRESENT
-DNV_SIGNAL_STRUCT_RLIM" ./NVIDIA-Linux-x86_64-185.18.31-pkg2.run

3)

./NVIDIA-Linux-x86_64-185.18.31-pkg2.run --extract-only

cd usr/src/nv

IGNORE_XEN_PRESENCE=y CC="gcc -DNV_VMAP_4_PRESENT -DNV_SIGNAL_STRUCT_RLIM"
make SYSSRC=/usr/src/kernels/2.6.30-rc3-enming.teo-tip module

cp nvidia.ko /lib/modules/2.6.30-rc3-enming.teo-tip/kernel/drivers/video

cd /lib/modules/2.6.30-rc3-enming.teo-tip/kernel/drivers/video

depmod -a

rmmod nvidia

lsmod | grep nvidia

modprobe nvidia

lsmod | grep nvidia

None of the above three scenarios I have attempted worked for me.

When I tried to load the newly compiled nvidia driver (against the Xen
Domain 0 paravirt operations kernel), it gives me the following message:

NVRM: loading NVIDIA UNIX x86_64 Kernel Module 185.18.31 Tue Jul 28 17:52:27
PDT 2009

NVRM: PAT configuration unsupported, falling back to MTRRs.

Any more ideas?

My home PC configuration:

Intel Pentium processor E6300 LGA775
Intel Desktop Board DQ45CB
BIOS Version 0093
8 GB DDR2-800
NVIDIA Geforce 8400 GS 512 MB DDR2 PCI Express x16

Fedora 11 64-bit Host operating system
Xen hypervisor 3.4.1
Jeremy Fitzhardinge's pv-ops dom 0 Xen kernel 2.6.30-rc3 and 2.6.31-rc6
Default shipping X server
mesa-lib upgraded

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
Alma Maters: Singapore Polytechnic, National University of Singapore



On 08/21/2009 12:23 AM, Teo En Ming (Zhang Enming) wrote:

This certainly looks promising. No need to apply any patch to NVIDIA
graphics drivers for it to be Xen-aware.

Link: http://www.nvnews.net/vbulletin/showthread.php?t=122900

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 9:21 PM
To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: RE: [Xen-devel] Vt-d not working with 3.4.1

<QUOTE drivesoslow>

SOLVED!

It was a kernel issue. The current Fedora 8 XEN kernel was the issue.
Since
there was not a newer one available I enabled the development repo and
installed the Fedora 9 XEN kernel that was in it. A couple of small hacks
with the drive source and I have a working NVIDIA 169.07 driver. I assume
when the next kernel is release for Fedora 8 it will resolve this issue.

Was:
Code:

kernel-xen-2.6.21-2952.fc8
kernel-xen-devel-2.6.21-2952.fc8

Now:
Code:

kernel-xen-2.6.21.7-2892.fc9
kernel-xen-devel-2.6.21.7-2892.fc9

</QUOTE>


Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 9:10 PM
To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: RE: [Xen-devel] Vt-d not working with 3.4.1

<QUOTE drivesoslow>

I was able to get it to compile and install no problem finally, but it
locks
up when I startx. I can compile it for non-XEN 64bit no problem. The
non-XEN
kernel is significantly newer (2.6.23 vs 2.6.21)

</QUOTE>

Link: http://www.nvnews.net/vbulletin/showthread.php?t=106395

Same problem as what I am getting.

I am using the NVidia Geforce 8 series drivers for 64-bit.

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 8:52 PM
To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: RE: [Xen-devel] Vt-d not working with 3.4.1

<QUOTE monz at nV News Forums>

The problem with video on Xen is that the driver needs to be Xen-aware.

</QUOTE>

Link: http://www.nvnews.net/vbulletin/showthread.php?t=60125

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 8:49 PM
To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: RE: [Xen-devel] Vt-d not working with 3.4.1

Will the Xen patch for NVIDIA graphics drivers mentioned in the below
opensuse.org link work with Xen 3.4.1 rc and Jeremy's pv-ops dom 0 kernel
2.6.30-rc3 or 2.6.31-rc6 under Fedora 11 64-bit? Please see the patch code
below.

<CODE>

diff -urN nv-1.0-9625/nv.c nv-1.0-9625-xenrt/nv.c
--- nv-1.0-9625/nv.c    2006-09-26 21:33:35.000000000 +0200
+++ nv-1.0-9625-xenrt/nv.c      2006-10-03 01:15:42.000000000 +0200
@@ -42,8 +42,26 @@

  int nv_pat_enabled = 0;

+/*
+ * disable PAT support if XEN or PREEMPT_RT is configured in kernel
+ */
+
+#if defined(CONFIG_XEN) || defined(CONFIG_PREEMPT_RT)
+static int nv_disable_pat = 1;
+#else
  static int nv_disable_pat = 0;
+#endif
+
+/*
+ * you can re-enable PAT support for PREEMPT_RT when applying
+ * "nv_disable_pat=0" as kernel parameter for the sake of slightly
+ * better 3D performance but at the expense of higher latencies.
+ * if XEN is configured, then PAT support can't be enabled!
+ */
+
+#if !defined(CONFIG_XEN)
  NV_MODULE_PARAMETER(nv_disable_pat);
+#endif

  #if defined(NVCPU_X86) || defined(NVCPU_X86_64)
  NvU64 __nv_supported_pte_mask = ~_PAGE_NX;
diff -urN nv-1.0-9625/nv-linux.h nv-1.0-9625-xenrt/nv-linux.h
--- nv-1.0-9625/nv-linux.h      2006-09-26 21:33:37.000000000 +0200
+++ nv-1.0-9625-xenrt/nv-linux.h        2006-10-03 01:15:42.000000000
+0200
@@ -226,7 +226,7 @@
   * tiny, and the kernel panics when it is exhausted. try to warn the user
that
   * they need to boost the size of their pool.
   */
-#if defined(CONFIG_SWIOTLB)&&    !defined(GFP_DMA32)
+#if defined(CONFIG_SWIOTLB)&&    !defined(GFP_DMA32)&&
  !defined(CONFIG_XEN)
  #define NV_SWIOTLB 1
  #endif

@@ -734,7 +734,10 @@
  #define NV_VM_INSERT_PAGE(vma, addr, page) \
      vm_insert_page(vma, addr, page)
  #endif
-#if defined(NV_REMAP_PFN_RANGE_PRESENT)
+#if defined(CONFIG_XEN)
+#define NV_REMAP_PAGE_RANGE(from, offset, x...) \
+    io_remap_pfn_range(vma, from, ((offset)>>    PAGE_SHIFT), x)
+#elif defined(NV_REMAP_PFN_RANGE_PRESENT)
  #define NV_REMAP_PAGE_RANGE(from, offset, x...) \
      remap_pfn_range(vma, from, ((offset)>>    PAGE_SHIFT), x)
  #elif defined(NV_REMAP_PAGE_RANGE_5_PRESENT)
@@ -746,6 +749,9 @@
  #define NV_REMAP_PAGE_RANGE(x...) remap_page_range(x)
  #endif

+#if !defined(CONFIG_XEN)
+#define phys_to_machine(x) x
+#endif

  #define NV_PGD_OFFSET(address, kernel, mm)              \
     ({                                                   \
diff -urN nv-1.0-9625/nv-vm.c nv-1.0-9625-xenrt/nv-vm.c
--- nv-1.0-9625/nv-vm.c 2006-09-26 21:33:37.000000000 +0200
+++ nv-1.0-9625-xenrt/nv-vm.c   2006-10-03 01:24:31.000000000 +0200
@@ -352,6 +352,9 @@

  static void nv_flush_caches(void)
  {
+#if defined(CONFIG_PREEMPT_RT)
+    if(!nv_pat_enabled) return;
+#endif
  #if defined(KERNEL_2_4)
      // for 2.4 kernels, just automatically flush the caches and
invalidate
tlbs
  #ifdef CONFIG_SMP
@@ -508,7 +511,7 @@
          page_ptr->phys_addr = phys_addr;
          page_ptr->page_count = NV_GET_PAGE_COUNT(page_ptr);
          page_ptr->virt_addr = virt_addr;
-        page_ptr->dma_addr = page_ptr->phys_addr;
+        page_ptr->dma_addr = phys_to_machine(page_ptr->phys_addr);

          /* lock the page for dma purposes */
          nv_lock_page(page_ptr);
diff -urN nv-1.0-9625/os-agp.c nv-1.0-9625-xenrt/os-agp.c
--- nv-1.0-9625/os-agp.c        2006-09-26 21:33:37.000000000 +0200
+++ nv-1.0-9625-xenrt/os-agp.c  2006-10-03 01:15:42.000000000 +0200
@@ -286,7 +286,7 @@

           page_ptr->phys_addr = (ptr->memory[i]&    PAGE_MASK);
           page_ptr->virt_addr = (unsigned long) __va(page_ptr->phys_addr);
-         page_ptr->dma_addr  = page_ptr->phys_addr;
+         page_ptr->dma_addr  = phys_to_machine(page_ptr->phys_addr);
      }

      return RM_OK;
diff -urN nv-1.0-9625/os-interface.c nv-1.0-9625-xenrt/os-interface.c
--- nv-1.0-9625/os-interface.c  2006-09-26 21:33:37.000000000 +0200
+++ nv-1.0-9625-xenrt/os-interface.c    2006-10-03 01:15:42.000000000
+0200
@@ -527,6 +527,7 @@
      MicroSeconds = MilliSeconds * 1000;
      tm_end.tv_usec = MicroSeconds;
      tm_end.tv_sec = 0;
+#if !defined(CONFIG_XEN)
      NV_TIMERADD(&tm_aux,&tm_end,&tm_end);

      /* do we have a full jiffie to wait? */
@@ -564,6 +565,7 @@
                  MicroSeconds = 0;
          } while ((jiffies = NV_USECS_TO_JIFFIES(MicroSeconds)) != 0);
      }
+#endif

      if (MicroSeconds>    1000)
      {

</CODE>

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 8:37 PM
To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: RE: [Xen-devel] Vt-d not working with 3.4.1

No wonder compiling official NVidia drivers against the Xen domain 0
paravirt operations kernel doesn't work. It causes my X server to show a
blank screen and freeze. NVidia drivers need to be patched for it to work
with Xen kernels.

http://en.opensuse.org/Use_Nvidia_driver_with_Xen

May I know where I can get the patch?

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Mr. Teo En
Ming
(Zhang Enming)
Sent: Thursday, August 20, 2009 1:06 AM
To: mad@xxxxxx
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
Subject: Re: [Xen-devel] Vt-d not working with 3.4.1

Dear All,

I can confirm that recompiling the Xen dom 0 paravirt-ops kernel using
gzip compression works. I can use the generated bzImage to boot up Domain
0.

If you attempt to compile the Xen Dom 0 pv-ops kernel using bzip2
compression, you won't be able to boot up Domain 0 using the generated
bzImage. You will get an ELF error and kernel panic on CPU 0.

As for X server, I haven't got much luck yet.

I was able to download and compile NVIDIA's kernel module directly from
official NVIDIA website for Fedora 11 Default shipping kernel. I was
able to start X server successfully and launch a desktop environment
using nvidia.ko.

However, if I want to compile the nvidia kernel module against Xen dom 0
pv_ops kernel sources, the kernel module compiles successfully, but when
I start X, the screen becomes blank and the whole system freezes there.




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

I am fairly sure that IGNORE_XEN_PRESENCE=y only makes the driver work
with a traditional (not pvops) patched xen kernel, I don't think it
will help to make the module compile or work with a pv_ops kernel.

Andy





_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

Re: [Xen-devel] Vt-d not working with 3.4.1