Re: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru support

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Thx for your answers !
I forgot about the primary/secondary difference.

From what i have seen so far it's probably not possible to automatically copy 
the bios of secondary card to use as bios in a hvm ?
I will try to load it from file, thx for the pointer.

(i'm using a nvidia card as primary/boot, and intent to keep it as xen/dom0 
console, and have a secondary ati which i want to passthrough to a hvm as 
primary)

--

Sander


Monday, December 13, 2010, 9:33:02 PM, you wrote:

> Hi Sander,

> I answer your questions (including the one from last email) below:

> 1. Your gfx is a bit old. Some of the tricks (such as MMINDEX values 0x4010) 
> aren't quite consistent across different generations of ATI gfx. I need to 
> look at the log myself. I will locate an HD3000 card to debug.
> 2. The goal of this patch is to passthru the primary gfx as THE ONLY GFX 
> inside guest VM (namely: my patch + set gfx_passthru=1 in guest config file). 
> Since you are passthru the on-board secondary gfx, I am not sure qemu-dm is 
> copying the correct VBIOS from address 0xc0000. One solution is to force 
> VBIOS been read from a file, which can be downloaded from 
> http://www.techpowerup.com/vgabios/.
> 3. Note that current Xen support passthru ATI gfx as a secondary gfx, even 
> without my patch. So if you don't care about primary/secondary, you probably 
> don't need this patch. Have you tried: existing Xen + passthru ATI without 
> gfx_passthru=1? As for as I have seen, it worked for many ATI gfx.

> Thanks,
> -Wei

> -----Original Message-----
> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx]
> Sent: Monday, December 13, 2010 2:18 PM
> To: Huang2, Wei
> Cc: Keir Fraser; Xen-devel; Ian Jackson; Kay, Allen M; Wang2, Wei; 
> djmagee@xxxxxxxxxxxx
> Subject: Re: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru support

> Hello Wei,

> It just occured to me, this is probably only working when trying to 
> passthrough the primary graphics card ?
> I'm trying to passthrough a secondary card, so the primary(boot) card still 
> has the xen/dom0 console.

> It could end up giving problems by loading the wrong/no bios in the hvmloader 
> ?

> Should passing though a secondary graphics card also be possible with this 
> patch ?

> --
> Sander

> Sunday, December 12, 2010, 7:19:08 AM, you wrote:

>> This patch isn't meant to be final version. The stubdom issue was known to 
>> me. Users can just compile qemu-dm and copy it to overwrite existing Xen 
>> installation for a quick testing.

>> Thanks,
>> -Wei
>> ________________________________________
>> From: Keir Fraser [keir.xen@xxxxxxxxx] On Behalf Of Keir Fraser 
>> [keir@xxxxxxx]
>> Sent: Saturday, December 11, 2010 9:38 AM
>> To: Sander Eikelenboom; Huang2, Wei
>> Cc: Xen-devel; Ian Jackson; Kay, Allen M; Wang2, Wei; djmagee@xxxxxxxxxxxx
>> Subject: Re: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru support

>> The patch would need some work to make it suitable for check in and get it
>> working for stubdom. At the very least the ioperm() calls would need
>> removing completely, or stubbing out for the stubdom build.

>>  -- Keir

>> On 11/12/2010 14:06, "Sander Eikelenboom" <linux@xxxxxxxxxxxxxx> wrote:

>>> Do i need any previous patches for this to work ?
>>>
>>> After applying it to xen-unstable, compiling xen results in:
>>>
>>> make[2]: Entering directory `/usr/src/new/xen-unstable.hg/extras/mini-os'
>>> [ -e include/xen ] || ln -sf ../../../xen/include/public include/xen
>>> [ -e include/mini-os ] || ln -sf . include/mini-os
>>> [ -e include/x86/mini-os ] || ln -sf . include/x86/mini-os
>>> make --directory=arch/x86
>>> OBJ_DIR=/usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/arch/x86 
>>> ||
>>> exit 1;
>>> make[3]: Entering directory
>>> `/usr/src/new/xen-unstable.hg/extras/mini-os/arch/x86'
>>> make[3]: Nothing to be done for `all'.
>>> make[3]: Leaving directory
>>> `/usr/src/new/xen-unstable.hg/extras/mini-os/arch/x86'
>>> ld -r -nostdlib
>>> -L/usr/src/new/xen-unstable.hg/stubdom/cross-root-x86_64/x86_64-xen-elf/lib
>>> -m elf_x86_64
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/arch/x86/x86_64.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os_app.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/blkfront.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/events.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/fbfront.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/fs-front.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/gntmap.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/gnttab.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/hypervisor.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/kernel.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lock.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/main.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mm.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/netfront.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/pcifront.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/sched.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/ctype.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/math.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/printf.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/stack_chk_fail.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/string.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/sys.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/xmalloc.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lib/xs.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/xenbus/xenbus.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/console/console.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/console/xencons_ring
>>> .o /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/lwip.a
>>> -L/usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/arch/x86 
>>> -lx86_64
>>> -lc -o /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o
>>> objcopy -w -G xenos_* -G _start
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o
>>> ld -nostdlib
>>> -L/usr/src/new/xen-unstable.hg/stubdom/cross-root-x86_64/x86_64-xen-elf/lib
>>> -m elf_x86_64 -T arch/x86/minios-x86_64.lds
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o  -o
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o: In
>>> function `ati_hw_out':
>>> /usr/src/new/xen-unstable.hg/stubdom/ioemu/hw/pt-graphics.c:82: undefined
>>> reference to `ioperm'
>>> /usr/src/new/xen-unstable.hg/stubdom/ioemu/hw/pt-graphics.c:84: undefined
>>> reference to `ioperm'
>>> /usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os.o: In
>>> function `ati_hw_in':
>>> /usr/src/new/xen-unstable.hg/stubdom/ioemu/hw/pt-graphics.c:72: undefined
>>> reference to `ioperm'
>>> /usr/src/new/xen-unstable.hg/stubdom/ioemu/hw/pt-graphics.c:74: undefined
>>> reference to `ioperm'
>>> make[2]: ***
>>> [/usr/src/new/xen-unstable.hg/stubdom/mini-os-x86_64-ioemu/mini-os] Error 1
>>> make[2]: Leaving directory `/usr/src/new/xen-unstable.hg/extras/mini-os'
>>> make[1]: *** [ioemu-stubdom] Error 2
>>> make[1]: Leaving directory `/usr/src/new/xen-unstable.hg/stubdom'
>>> make: *** [install-stubdom] Error 2
>>>
>>> Don't know why the include of sys/io.h doesn't seem to work
>>> --
>>>
>>> Sander
>>>
>>> Saturday, December 11, 2010, 12:40:19 AM, you wrote:
>>>
>>>> Hi,
>>>
>>>> The attached patch supports dynamic detection of BARs (both MMIO and
>>>> PIO). Hopefully it can alleviate some of the issues you saw with ATI gfx
>>>> passthru. Please let me know whether it works better for you.
>>>
>>>> Thanks,
>>>> -Wei
>>>
>>>> On 12/07/2010 04:00 AM, Pasi Kärkkäinen wrote:
>>>>> On Wed, Oct 13, 2010 at 04:37:10PM -0500, Huang2, Wei wrote:
>>>>>>     Sorry, hold on a second. This fix seems corrupt my pci config space
>>>>>> after
>>>>>>     several runs.
>>>>>>
>>>>> Hello,
>>>>>
>>>>> Any updates to these patches? Many users have been asking about amd/ati 
>>>>> vga
>>>>> passthru stuff..
>>>>>
>>>>> -- Pasi
>>>>>
>>>>>>
>>>>>>     -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>>>>>     [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Huang2, 
>>>>>> Wei
>>>>>>     Sent: Wednesday, October 13, 2010 3:47 PM
>>>>>>     To: djmagee@xxxxxxxxxxxx; Kay, Allen M; Ian Jackson
>>>>>>     Cc: Wang2, Wei; Xen-devel
>>>>>>     Subject: RE: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru
>>>>>>     support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Allen and Doug,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Could you fix the following line in save_pci_conf_space() of
>>>>>>     tools/python/xen/util/pci.py?
>>>>>>
>>>>>>
>>>>>>
>>>>>>             "for i in range(0, 256, 4):" to "for i in range(0, 512, 4):"
>>>>>>
>>>>>>
>>>>>>
>>>>>>     This solves my black screen issue. Please let me know the results.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Thanks,
>>>>>>
>>>>>>     -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: djmagee@xxxxxxxxxxxx [mailto:djmagee@xxxxxxxxxxxx]
>>>>>>     Sent: Friday, October 08, 2010 10:53 AM
>>>>>>     To: Huang2, Wei; Kay, Allen M; Ian Jackson
>>>>>>     Cc: Xen-devel; Wang2, Wei
>>>>>>     Subject: RE: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru
>>>>>>     support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Wei,
>>>>>>
>>>>>>                     These were guests that had never seen the catalyst
>>>>>> driver
>>>>>>     before.  I booted each three with the graphics device and usb devices
>>>>>>     assigned, they worked fine using the basic VGA driver, then installed
>>>>>>     fresh Catalyst 10.9, rebooted, and each one crashed.
>>>>>>
>>>>>>
>>>>>>
>>>>>>                     Let me know if there's any other info I can provide
>>>>>> that
>>>>>>     will help you debug.  The motherboard is a DQ45CB, running 
>>>>>> xen-unstable
>>>>>>     c/s 22155 using `dom0_mem=768M iommu=1' on the grub line, and using
>>>>>> pvops
>>>>>>     Dom0 from stable-2.6.32.x commit 179eca50.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Doug Magee
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: Huang2, Wei [mailto:Wei.Huang2@xxxxxxx]
>>>>>>     Sent: Friday, October 08, 2010 11:41 AM
>>>>>>     To: djmagee@xxxxxxxxxxxx; Kay, Allen M; Ian Jackson
>>>>>>     Cc: Xen-devel; Wang2, Wei
>>>>>>     Subject: RE: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru
>>>>>>     support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Doug,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Regarding Catalyst driver, we never saw guest crashing. Was it a 
>>>>>> fresh
>>>>>>     Catalyst installation or an existing guest image? We felt this VBIOS
>>>>>> patch
>>>>>>     might not provide all necessary resources to driver, which got upset.
>>>>>>     Currently we are debugging it with our driver team and will let you
>>>>>> know
>>>>>>     the update.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Thanks,
>>>>>>
>>>>>>     -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: djmagee@xxxxxxxxxxxx [mailto:djmagee@xxxxxxxxxxxx]
>>>>>>     Sent: Friday, October 08, 2010 9:41 AM
>>>>>>     To: Huang2, Wei; Kay, Allen M; Ian Jackson
>>>>>>     Cc: Xen-devel
>>>>>>     Subject: RE: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru
>>>>>>     support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Wei,
>>>>>>
>>>>>>                     I've tested with a Radeon 4770 and it the VBIOS works
>>>>>>     without a problem, through many guest (re)boots, so it seems pretty
>>>>>>     solid.  I tested a linux guest (fairly standard Fedora 12), Windows 
>>>>>> XP,
>>>>>>     and Windows 7 (without accelerated drivers).  The open radeon linux
>>>>>> driver
>>>>>>     works fine.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     The Catalyst driver fails in Windows XP (STOP 0x000000EA, thread 
>>>>>> stuck
>>>>>> in
>>>>>>     device driver), windows 7 (STOP 0x00000116, driver fails to properly
>>>>>>     reset?), and linux (total system freeze).  This seems to be different
>>>>>> than
>>>>>>     the `Blank Screen' problem you report, as the driver is clearly not
>>>>>>     functioning properly.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Doug Magee
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
>>>>>>     [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Huang2, 
>>>>>> Wei
>>>>>>     Sent: Friday, October 08, 2010 9:57 AM
>>>>>>     To: Kay, Allen M; Ian Jackson
>>>>>>     Cc: Xen-devel
>>>>>>     Subject: [Xen-devel] RE: [RFC][QEMU] ATI graphics VBIOS passthru
>>>>>> support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Allen,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Yes, Catalyst driver is the one from public website. The driver still
>>>>>> has
>>>>>>     a  minor issue with this VBIOS patch. The purpose of submitting VBIOS
>>>>>>     patch is to get community feedbacks. After we figure out the root 
>>>>>> cause
>>>>>> of
>>>>>>     black screen, I will formally submit a patch for inclusion.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Thanks,
>>>>>>
>>>>>>     -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: Kay, Allen M [mailto:allen.m.kay@xxxxxxxxx]
>>>>>>     Sent: Friday, October 08, 2010 2:21 AM
>>>>>>     To: Huang2, Wei; Ian Jackson
>>>>>>     Cc: Xen-devel
>>>>>>     Subject: RE: [RFC][QEMU] ATI graphics VBIOS passthru support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Wei,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Is Catalyst driver the one on AMD website?  I think that's what I 
>>>>>> have
>>>>>> in
>>>>>>     my win7 guest and it matches the symptom you are describing.  "lspci"
>>>>>>     reports my ATI card is a V5700 - although it says v3750 on the box.
>>>>>> Where
>>>>>>     can I get a working driver?
>>>>>>
>>>>>>
>>>>>>
>>>>>>     The patch looks reasonable to me in general.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Allen
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: Huang2, Wei [mailto:Wei.Huang2@xxxxxxx]
>>>>>>     Sent: Thursday, October 07, 2010 9:06 PM
>>>>>>     To: Kay, Allen M; Ian Jackson
>>>>>>     Cc: Xen-devel
>>>>>>     Subject: RE: [RFC][QEMU] ATI graphics VBIOS passthru support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Allen,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Thanks for testing it out. We have tested this patch with Radeon 
>>>>>> 4850,
>>>>>>     4870, FirePro V5700 and FirePro M5800. Unfortunately we don't have
>>>>>> V3750
>>>>>>     at hand. It is very possible this patch isn't compatible with V3750. 
>>>>>> We
>>>>>>     will try to get hold of one for debugging. For graphics which work 
>>>>>> with
>>>>>>     this path, users should be able to get rid of emulated gfx (such as
>>>>>>     Cirrus). I have successfully installed a Windows guest VM using this
>>>>>>     patch.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     I also want to point out that there is still an issue. Users will 
>>>>>> see a
>>>>>>     black screen after installing Catalyst driver. Even though the screen
>>>>>>     appears to be black, the driver is actually functioning correctly
>>>>>> (3DMark
>>>>>>     can be run with external monitor). Our driver team is currently
>>>>>> debugging
>>>>>>     it and they believe this is easy to fix.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     What is your opinion on this patch (and the solution) in general?
>>>>>>
>>>>>>
>>>>>>
>>>>>>     -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: Kay, Allen M [mailto:allen.m.kay@xxxxxxxxx]
>>>>>>     Sent: Thursday, October 07, 2010 6:58 PM
>>>>>>     To: Huang2, Wei; Ian Jackson
>>>>>>     Cc: Xen-devel
>>>>>>     Subject: RE: [RFC][QEMU] ATI graphics VBIOS passthru support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Wei,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     This patch did not cause any problems with Intel IGD passthrough for
>>>>>> me.
>>>>>>     However, the monitor remained blank if I pass through ATI Firepro 
>>>>>> V3750
>>>>>>     either as the primary display device or the secondary device
>>>>>>     (gfx_passthru=1/0).  Passing it through as the secondary device used 
>>>>>> to
>>>>>>     work.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Have you tested the patch with this graphics card?
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Allen
>>>>>>
>>>>>>
>>>>>>
>>>>>>     From: Huang2, Wei [mailto:Wei.Huang2@xxxxxxx]
>>>>>>     Sent: Thursday, October 07, 2010 9:57 AM
>>>>>>     To: Ian Jackson
>>>>>>     Cc: Xen-devel; Kay, Allen M
>>>>>>     Subject: [RFC][QEMU] ATI graphics VBIOS passthru support
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Hi Ian,
>>>>>>
>>>>>>
>>>>>>
>>>>>>     There have been a lot of interest on gfx passthru recently. This 
>>>>>> patch
>>>>>>     enables ATI VBIOS in passthru mode. The guest VM system BIOS 
>>>>>> (including
>>>>>>     Windows boot logo) can now show in passthru screen. We have tested 
>>>>>> with
>>>>>>     various Windows and Linux guest VMs. Please help review it. We are 
>>>>>> also
>>>>>>     looking forward to comments and suggestions from Xen community users.
>>>>>>
>>>>>>
>>>>>>
>>>>>>     Signed-off-by: Wei Huang<wei.huang2@xxxxxxx>
>>>>>>
>>>>>>     Signed-off-by: Wei Wang<wei.wang2@xxxxxxx>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Xen-devel mailing list
>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>> http://lists.xensource.com/xen-devel
>>>>>
>>>
>>>
>>>







> --
> Best regards,
>  Sander                            mailto:linux@xxxxxxxxxxxxxx






-- 
Best regards,
 Sander                            mailto:linux@xxxxxxxxxxxxxx


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel