Konrad,
On Mon, Jun 27, 2011 at 10:11, Konrad Rzeszutek Wilk
<konrad.wilk@xxxxxxxxxx> wrote:
> On Sun, Jun 26, 2011 at 09:35:17PM -0600, Alex Merritt wrote:
>> Hello,
>>
>> I'm looking to enable the NVIDIA CUDA driver/runtime stack to work in
>> Dom0 on Xen. I've contacted NVIDIA through various capacities and have
>
> I've been running the NVIDIA binary driver with 2.6.39 / 3.0 kernel.
To avoid confusion here, with 2.6.39* I have been able to successfully
build, install and load the NVIDIA binary driver. Executing the SDK
applications fail (except for deviceQuery), telling me devices are not
available or don't exist. I had to manually create the device files
(/dev/nvidiactl, /dev/nvidia[0-9]) and while in runlevel 3 I had to
create a persistent client with 'nvidia-smi -pm 1' to avoid the
initial driver state caching so that applications don't have an
initial slow-down when the CUDA runtime library opens the devices
files for the first time. Have you been able to do so otherwise? I'm
not sure what exactly you mean by "running the NVIDIA binary driver".
However, installing the driver on Dom0/3.0.0-rc5 fails for me. The
build process executes a configuration test (conftest.sh:1488 after
extracting the driver sources) to determine the which Makefile to use,
then fails.
conftest.sh:1517 in <extracted nvidia sources>/kernel/ removing the
stdout/stderr redirection:
<src>/include/linux/rcupdate.h: In function '__kfree_rcu':
<src>/include/linux/rcupdate.h:822: error: size of array 'type name' is negative
--------------------------------
$ sh devdriver_4.0_linux_64_270.41.19.run -x
$ cd NVIDIA-Linux-x86_64-270.41.19/kernel
$ make module
If you are using a Linux 2.4 kernel, please make sure
you either have configured kernel sources matching your
kernel or the correct set of kernel headers installed
on your system.
If you are using a Linux 2.6 kernel, please make sure
you have configured kernel sources matching your kernel
installed on your system. If you specified a separate
output directory using either the "KBUILD_OUTPUT" or
the "O" KBUILD parameter, make sure to specify this
directory with the SYSOUT environment variable or with
the equivalent nvidia-installer command line option.
Depending on where and how the kernel sources (or the
kernel headers) were installed, you may need to specify
their location with the SYSSRC environment variable or
the equivalent nvidia-installer command line option.
*** Unable to determine the target kernel version. ***
make: *** [select_makefile] Error 1
--------------------------------
This cannot be true as I just built it and booted into it.
/lib/modules/$(uname -r)/build points to the correct sources. You did
not encounter this hiccup?
>
>> gotten replies essentially saying they cannot provide assistance, and
>> have been following the nvnews.com forums. However, now that I have
>> IOMMU-capable processors (with VT-d) and a version of Xen which can
>> successfully program this hardware, I am interested to determine if it
>> is possible to program the IOMMU *for* Dom0 (i.e. the target is Dom0,
>> not an HVM/pvops guest).
>>
>> I have successfully been able to launch an HVM guest, passing a few
>> GPUs through. Installing an unmodified developer driver from NVIDIA
>> works using the standard method (no SYSSRC=/-OUT= or anything) and
>> CUDA applications execute as expected. Can this be done for Dom0,
>> achieving the same result without requiring any involvement from
>> NVIDIA or modifications to the driver?
>
> It should be no trouble.. albeit you might need to update the Linux
> kernel to 2.6.39 or 3.0 to take advantage of the 1-1 P2M mapping code.
I will look into this once the module can build properly...
>
>>
>> My immediate interest is more to see if it "can be done" via a hack or
>> something, not necessarily whether it would make sense for Xen to
>> support this in the future. My goal with this email is to get feedback
>> on two fronts: 1) is there a limitation due to the architecture,
>> meaning that as Dom0 is pvops it cannot use VT-d, or is it a
>> implementation addition to Xen that would be needed? 2) If the latter
>> is true (programming needed), how much effort (code/time) would
>> you/anyone estimate it would take to enable this to work, and could
>> you provide some starting pointers for me to do so? I am untrained
>> when it comes to the Xen sourcecode.
>>
>> I'm using Xen 4.1.1 and pv-ops linux (not upstream) 2.6.32.40 on an
>> Intel X5660 with a Tylersburg chipset. My host OS is Fedora 13, but
>> that needn't be static.
>>
>> I originally asked this on xen-users, but was informed this mailing
>> list would be better suited:
>> http://lists.xensource.com/archives/html/xen-users/2011-06/msg00451.html
>>
>> Many thanks in advance!
>> Alex
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home