WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Vt-d not working with 3.4.1

To: enming.teo@xxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Vt-d not working with 3.4.1
From: Andrew Lyon <andrew.lyon@xxxxxxxxx>
Date: Thu, 20 Aug 2009 19:03:18 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, weidong.han@xxxxxxxxx, mad@xxxxxx, "Bonenkamp, Ralf" <ralf.bonenkamp@xxxxxxxx>
Delivery-date: Thu, 20 Aug 2009 11:03:52 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=svB1VFaMAyMEIY5NMeRLRLfOigFOaM9HsTlOexkrrM0=; b=J5yvQN+XFuIJP5fH1wBPNaUb8JjdtjTN3FajK+WZ8mLL915wf2yNg8DXj/SfCKzfqo p30rkXbc40vglx6upKolxTHsIgZjVUjrxnP3UG/LqBo1PAqvEtGpOOHKxznxdtgUn6QH I4W+b4T7k8mOe+0NKbvzDbgJYD3DTeOZngG3U=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=KeuTC6ykRvtmkXtvk4sTaXYVQI7MbJWW+QT2df9+ck8cOIQAVABVN4hZPBz7tbbAF9 qowyVEq9aQnVWwSwm/c8U/x42cnZpoDAPc4yRy0OGMxkLlWB7jLEksaMr4W0FtCfU/VB RPgkAkLM7DUeqwUiHT4PULI4HO3y8NNtkLHps=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4A8D7C9E.30503@xxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <833F02C491734E2BAA6CB4265638A47A@ASOITIS16> <4A8D7C9E.30503@xxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Thu, Aug 20, 2009 at 5:41 PM, Mr. Teo En Ming (Zhang
Enming)<enming.teo@xxxxxxxxxxxxxxx> wrote:
> Dear All,
>
> I followed the XEN - NVIDIA - STEP BY STEP instructions at nV news forums
> but I still couldn't get X to work under Xen pv-ops domain 0 kernel. It
> still shows me a freezed blank screen. I couldn't switch to any tty (1 to 7
> inclusive).
>
> Here are the scenarios I have tried:
>
> 1)     IGNORE_XEN_PRESENCE=y ./NVIDIA-Linux-x86_64-185.18.31-pkg2.run
>
> 2)     IGNORE_XEN_PRESENCE=y CC="gcc -DNV_VMAP_4_PRESENT
> -DNV_SIGNAL_STRUCT_RLIM" ./NVIDIA-Linux-x86_64-185.18.31-pkg2.run
>
> 3)
>
> ./NVIDIA-Linux-x86_64-185.18.31-pkg2.run --extract-only
>
> cd usr/src/nv
>
> IGNORE_XEN_PRESENCE=y CC="gcc -DNV_VMAP_4_PRESENT -DNV_SIGNAL_STRUCT_RLIM"
> make SYSSRC=/usr/src/kernels/2.6.30-rc3-enming.teo-tip module
>
> cp nvidia.ko /lib/modules/2.6.30-rc3-enming.teo-tip/kernel/drivers/video
>
> cd /lib/modules/2.6.30-rc3-enming.teo-tip/kernel/drivers/video
>
> depmod -a
>
> rmmod nvidia
>
> lsmod | grep nvidia
>
> modprobe nvidia
>
> lsmod | grep nvidia
>
> None of the above three scenarios I have attempted worked for me.
>
> When I tried to load the newly compiled nvidia driver (against the Xen
> Domain 0 paravirt operations kernel), it gives me the following message:
>
> NVRM: loading NVIDIA UNIX x86_64 Kernel Module 185.18.31 Tue Jul 28 17:52:27
> PDT 2009
>
> NVRM: PAT configuration unsupported, falling back to MTRRs.
>
> Any more ideas?
>
> My home PC configuration:
>
> Intel Pentium processor E6300 LGA775
> Intel Desktop Board DQ45CB
> BIOS Version 0093
> 8 GB DDR2-800
> NVIDIA Geforce 8400 GS 512 MB DDR2 PCI Express x16
>
> Fedora 11 64-bit Host operating system
> Xen hypervisor 3.4.1
> Jeremy Fitzhardinge's pv-ops dom 0 Xen kernel 2.6.30-rc3 and 2.6.31-rc6
> Default shipping X server
> mesa-lib upgraded
>
> --
> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
> BEng(Hons)(Mechanical Engineering)
> Technical Support Engineer
> Information Technology Department
> Asiasoft Online Pte Ltd
> Tampines Central 1 #04-01 Tampines Plaza
> Singapore 529541
> Republic of Singapore
> Mobile: +65-9648-9798
> MSN: teoenming@xxxxxxxxxxx
> Alma Maters: Singapore Polytechnic, National University of Singapore
>
>
>
> On 08/21/2009 12:23 AM, Teo En Ming (Zhang Enming) wrote:
>>
>> This certainly looks promising. No need to apply any patch to NVIDIA
>> graphics drivers for it to be Xen-aware.
>>
>> Link: http://www.nvnews.net/vbulletin/showthread.php?t=122900
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>>
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 9:21 PM
>> To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: RE: [Xen-devel] Vt-d not working with 3.4.1
>>
>> <QUOTE drivesoslow>
>>
>> SOLVED!
>>
>> It was a kernel issue. The current Fedora 8 XEN kernel was the issue.
>> Since
>> there was not a newer one available I enabled the development repo and
>> installed the Fedora 9 XEN kernel that was in it. A couple of small hacks
>> with the drive source and I have a working NVIDIA 169.07 driver. I assume
>> when the next kernel is release for Fedora 8 it will resolve this issue.
>>
>> Was:
>> Code:
>>
>> kernel-xen-2.6.21-2952.fc8
>> kernel-xen-devel-2.6.21-2952.fc8
>>
>> Now:
>> Code:
>>
>> kernel-xen-2.6.21.7-2892.fc9
>> kernel-xen-devel-2.6.21.7-2892.fc9
>>
>> </QUOTE>
>>
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>>
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 9:10 PM
>> To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: RE: [Xen-devel] Vt-d not working with 3.4.1
>>
>> <QUOTE drivesoslow>
>>
>> I was able to get it to compile and install no problem finally, but it
>> locks
>> up when I startx. I can compile it for non-XEN 64bit no problem. The
>> non-XEN
>> kernel is significantly newer (2.6.23 vs 2.6.21)
>>
>> </QUOTE>
>>
>> Link: http://www.nvnews.net/vbulletin/showthread.php?t=106395
>>
>> Same problem as what I am getting.
>>
>> I am using the NVidia Geforce 8 series drivers for 64-bit.
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>>
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 8:52 PM
>> To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: RE: [Xen-devel] Vt-d not working with 3.4.1
>>
>> <QUOTE monz at nV News Forums>
>>
>> The problem with video on Xen is that the driver needs to be Xen-aware.
>>
>> </QUOTE>
>>
>> Link: http://www.nvnews.net/vbulletin/showthread.php?t=60125
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 8:49 PM
>> To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: RE: [Xen-devel] Vt-d not working with 3.4.1
>>
>> Will the Xen patch for NVIDIA graphics drivers mentioned in the below
>> opensuse.org link work with Xen 3.4.1 rc and Jeremy's pv-ops dom 0 kernel
>> 2.6.30-rc3 or 2.6.31-rc6 under Fedora 11 64-bit? Please see the patch code
>> below.
>>
>> <CODE>
>>
>> diff -urN nv-1.0-9625/nv.c nv-1.0-9625-xenrt/nv.c
>> --- nv-1.0-9625/nv.c    2006-09-26 21:33:35.000000000 +0200
>> +++ nv-1.0-9625-xenrt/nv.c      2006-10-03 01:15:42.000000000 +0200
>> @@ -42,8 +42,26 @@
>>
>>  int nv_pat_enabled = 0;
>>
>> +/*
>> + * disable PAT support if XEN or PREEMPT_RT is configured in kernel
>> + */
>> +
>> +#if defined(CONFIG_XEN) || defined(CONFIG_PREEMPT_RT)
>> +static int nv_disable_pat = 1;
>> +#else
>>  static int nv_disable_pat = 0;
>> +#endif
>> +
>> +/*
>> + * you can re-enable PAT support for PREEMPT_RT when applying
>> + * "nv_disable_pat=0" as kernel parameter for the sake of slightly
>> + * better 3D performance but at the expense of higher latencies.
>> + * if XEN is configured, then PAT support can't be enabled!
>> + */
>> +
>> +#if !defined(CONFIG_XEN)
>>  NV_MODULE_PARAMETER(nv_disable_pat);
>> +#endif
>>
>>  #if defined(NVCPU_X86) || defined(NVCPU_X86_64)
>>  NvU64 __nv_supported_pte_mask = ~_PAGE_NX;
>> diff -urN nv-1.0-9625/nv-linux.h nv-1.0-9625-xenrt/nv-linux.h
>> --- nv-1.0-9625/nv-linux.h      2006-09-26 21:33:37.000000000 +0200
>> +++ nv-1.0-9625-xenrt/nv-linux.h        2006-10-03 01:15:42.000000000
>> +0200
>> @@ -226,7 +226,7 @@
>>   * tiny, and the kernel panics when it is exhausted. try to warn the user
>> that
>>   * they need to boost the size of their pool.
>>   */
>> -#if defined(CONFIG_SWIOTLB)&&  !defined(GFP_DMA32)
>> +#if defined(CONFIG_SWIOTLB)&&  !defined(GFP_DMA32)&&
>>  !defined(CONFIG_XEN)
>>  #define NV_SWIOTLB 1
>>  #endif
>>
>> @@ -734,7 +734,10 @@
>>  #define NV_VM_INSERT_PAGE(vma, addr, page) \
>>      vm_insert_page(vma, addr, page)
>>  #endif
>> -#if defined(NV_REMAP_PFN_RANGE_PRESENT)
>> +#if defined(CONFIG_XEN)
>> +#define NV_REMAP_PAGE_RANGE(from, offset, x...) \
>> +    io_remap_pfn_range(vma, from, ((offset)>>  PAGE_SHIFT), x)
>> +#elif defined(NV_REMAP_PFN_RANGE_PRESENT)
>>  #define NV_REMAP_PAGE_RANGE(from, offset, x...) \
>>      remap_pfn_range(vma, from, ((offset)>>  PAGE_SHIFT), x)
>>  #elif defined(NV_REMAP_PAGE_RANGE_5_PRESENT)
>> @@ -746,6 +749,9 @@
>>  #define NV_REMAP_PAGE_RANGE(x...) remap_page_range(x)
>>  #endif
>>
>> +#if !defined(CONFIG_XEN)
>> +#define phys_to_machine(x) x
>> +#endif
>>
>>  #define NV_PGD_OFFSET(address, kernel, mm)              \
>>     ({                                                   \
>> diff -urN nv-1.0-9625/nv-vm.c nv-1.0-9625-xenrt/nv-vm.c
>> --- nv-1.0-9625/nv-vm.c 2006-09-26 21:33:37.000000000 +0200
>> +++ nv-1.0-9625-xenrt/nv-vm.c   2006-10-03 01:24:31.000000000 +0200
>> @@ -352,6 +352,9 @@
>>
>>  static void nv_flush_caches(void)
>>  {
>> +#if defined(CONFIG_PREEMPT_RT)
>> +    if(!nv_pat_enabled) return;
>> +#endif
>>  #if defined(KERNEL_2_4)
>>      // for 2.4 kernels, just automatically flush the caches and
>> invalidate
>> tlbs
>>  #ifdef CONFIG_SMP
>> @@ -508,7 +511,7 @@
>>          page_ptr->phys_addr = phys_addr;
>>          page_ptr->page_count = NV_GET_PAGE_COUNT(page_ptr);
>>          page_ptr->virt_addr = virt_addr;
>> -        page_ptr->dma_addr = page_ptr->phys_addr;
>> +        page_ptr->dma_addr = phys_to_machine(page_ptr->phys_addr);
>>
>>          /* lock the page for dma purposes */
>>          nv_lock_page(page_ptr);
>> diff -urN nv-1.0-9625/os-agp.c nv-1.0-9625-xenrt/os-agp.c
>> --- nv-1.0-9625/os-agp.c        2006-09-26 21:33:37.000000000 +0200
>> +++ nv-1.0-9625-xenrt/os-agp.c  2006-10-03 01:15:42.000000000 +0200
>> @@ -286,7 +286,7 @@
>>
>>           page_ptr->phys_addr = (ptr->memory[i]&  PAGE_MASK);
>>           page_ptr->virt_addr = (unsigned long) __va(page_ptr->phys_addr);
>> -         page_ptr->dma_addr  = page_ptr->phys_addr;
>> +         page_ptr->dma_addr  = phys_to_machine(page_ptr->phys_addr);
>>      }
>>
>>      return RM_OK;
>> diff -urN nv-1.0-9625/os-interface.c nv-1.0-9625-xenrt/os-interface.c
>> --- nv-1.0-9625/os-interface.c  2006-09-26 21:33:37.000000000 +0200
>> +++ nv-1.0-9625-xenrt/os-interface.c    2006-10-03 01:15:42.000000000
>> +0200
>> @@ -527,6 +527,7 @@
>>      MicroSeconds = MilliSeconds * 1000;
>>      tm_end.tv_usec = MicroSeconds;
>>      tm_end.tv_sec = 0;
>> +#if !defined(CONFIG_XEN)
>>      NV_TIMERADD(&tm_aux,&tm_end,&tm_end);
>>
>>      /* do we have a full jiffie to wait? */
>> @@ -564,6 +565,7 @@
>>                  MicroSeconds = 0;
>>          } while ((jiffies = NV_USECS_TO_JIFFIES(MicroSeconds)) != 0);
>>      }
>> +#endif
>>
>>      if (MicroSeconds>  1000)
>>      {
>>
>> </CODE>
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>>
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Teo En Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 8:37 PM
>> To: enming.teo@xxxxxxxxxxxxxxx; mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: RE: [Xen-devel] Vt-d not working with 3.4.1
>>
>> No wonder compiling official NVidia drivers against the Xen domain 0
>> paravirt operations kernel doesn't work. It causes my X server to show a
>> blank screen and freeze. NVidia drivers need to be patched for it to work
>> with Xen kernels.
>>
>> http://en.opensuse.org/Use_Nvidia_driver_with_Xen
>>
>> May I know where I can get the patch?
>>
>> Regards,
>>
>> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
>> BEng(Hons)(Mechanical Engineering)
>> Technical Support Engineer
>> Information Technology Department
>> Asiasoft Online Pte Ltd
>> Tampines Central 1 #04-01 Tampines Plaza
>> Singapore 529541
>> Republic of Singapore
>> Mobile: +65-9648-9798
>> MSN: teoenming@xxxxxxxxxxx
>> -----Original Message-----
>> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Mr. Teo En
>> Ming
>> (Zhang Enming)
>> Sent: Thursday, August 20, 2009 1:06 AM
>> To: mad@xxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; weidong.han@xxxxxxxxx; 'Bonenkamp,Ralf'
>> Subject: Re: [Xen-devel] Vt-d not working with 3.4.1
>>
>> Dear All,
>>
>> I can confirm that recompiling the Xen dom 0 paravirt-ops kernel using
>> gzip compression works. I can use the generated bzImage to boot up Domain
>> 0.
>>
>> If you attempt to compile the Xen Dom 0 pv-ops kernel using bzip2
>> compression, you won't be able to boot up Domain 0 using the generated
>> bzImage. You will get an ELF error and kernel panic on CPU 0.
>>
>> As for X server, I haven't got much luck yet.
>>
>> I was able to download and compile NVIDIA's kernel module directly from
>> official NVIDIA website for Fedora 11 Default shipping kernel. I was
>> able to start X server successfully and launch a desktop environment
>> using nvidia.ko.
>>
>> However, if I want to compile the nvidia kernel module against Xen dom 0
>> pv_ops kernel sources, the kernel module compiles successfully, but when
>> I start X, the screen becomes blank and the whole system freezes there.
>>
>>
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>

I am fairly sure that IGNORE_XEN_PRESENCE=y only makes the driver work
with a traditional (not pvops) patched xen kernel, I don't think it
will help to make the module compile or work with a pv_ops kernel.

Andy

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>