Hi Andreas,
Thanks for asking, I figure it is time to start organising some of my notes.
This is what I've got so far, perhaps you'll find something in them?
*****************************************************************************************************************
Using PCI devices in a driver domain (a privileged domU?)
====================================================================
The examples below use the following domains.
wench: dom0 on P3/650 on Asus P2BVM, 512MB RAM
"Production" Xen Server
brutus: dom0 on P2/350 on Asus P2B, 192MB ECC RAM
"Development" Xen Server
BluetoothProxy: domU on wench (in use)
XenFirewall: domU on brutus (WIP)
These notes are mostly to clean up my own documentation, and they
_hopefully_ contain
all the information I need to repeat the process.
It is not necessarily HOWTO CREATE DRIVER DOMAINS, but how I got mine to
work.
1. Hide the device from the hypervisor (dom0)
EXAMPLE A: brutus
-----------------------------
This machine has a hidden USB Host Controller, and two network cards,
(the second) one being hidden from brutus.
It has a custom initrd as the root is on LVM.
brutus:~# lspci
0000:00:00.0 Host bridge: Intel Corp. 440BX/ZX/DX - 82443BX/ZX/DX Host
bridge (rev 03)
0000:00:01.0 PCI bridge: Intel Corp. 440BX/ZX/DX - 82443BX/ZX/DX AGP
bridge (rev 03)
0000:00:04.0 ISA bridge: Intel Corp. 82371AB/EB/MB PIIX4 ISA (rev 02)
0000:00:04.1 IDE interface: Intel Corp. 82371AB/EB/MB PIIX4 IDE (rev 01)
0000:00:04.3 Bridge: Intel Corp. 82371AB/EB/MB PIIX4 ACPI (rev 02)
0000:00:0b.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX
[Cyclone] (rev 30)
0000:01:00.0 VGA compatible controller: Silicon Integrated Systems [SiS]
86C326 5598/6326 (rev 0b)
brutus:~# tail /boot/grub/menu.lst
title Debian Xen
root (hd0,0)
kernel /xen-2.0-testing.gz dom0_mem=81920 root=/dev/ram0
lvm2root=/dev/mapper/vg0-root physdev_dom0_hide=(00:0c.0)(00:04:2) ro
console=tty0 noreboot
module /vmlinuz-2.6.11.12-xen0 root=/dev/mapper/vg0-root
console=tty0 ro noreboot
module /initrd.img-2.6.11.12-xen0
savedefault
boot
EXAMPLE B: wench
------------------------------
This machine has a hidden USB host controller
wench:~# lspci
0000:00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX
Host bridge (rev 03)
0000:00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX
AGP bridge (rev 03)
0000:00:06.0 Multimedia audio controller: ESS Technology ES1969 Solo-1
Audiodrive (rev 01)
0000:00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 02)
0000:00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE
(rev 01)
0000:00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
0000:00:0b.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX
[Cyclone] (rev 64)
0000:01:00.0 VGA compatible controller: ATI Technologies Inc 3D Rage IIC
AGP (rev 7a)
wench:~# tail /boot/grub/menu.lst
title Xen 2 - Kernel 2.6.11.12
root (hd0,0)
#kernel /xen-2.0-testing.gz dom0_mem=131072 root=/dev/hda3 ro
console=tty0
kernel /xen-2.0-testing.gz dom0_mem=131072 root=/dev/hda3 ro
console=tty0 physdev_dom0_hide=(00:07.2)
# Marcus appends phys_hide option so BluetoothProxy domU
can have USB Host Controller
module /vmlinuz-2.6.11.12-xen0 root=/dev/hda3 ro console=tty0
savedefault
boot
====================================================================
2. Configure kernel (& modules) for the driver domain
EXAMPLE A: Firewall on brutus
----------------------------------------------
brutus:~# egrep "XEN|PCI|NET|VORTEX"
/boot/config-2.6.11.12-XenFirewall | grep -v "#"
CONFIG_XEN=y
CONFIG_ARCH_XEN=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_WRITABLE_PAGETABLES=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XENARCH="i386"
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_LEGACY_PROC=y
CONFIG_PCI_NAMES=y
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_NET=y
CONFIG_INET=y
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_INET_TUNNEL=m
CONFIG_NETFILTER=y
CONFIG_BRIDGE_NETFILTER=y
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_NET_CLS_IND=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NETDEVICES=y
CONFIG_NET_ETHERNET=y
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_NET_PCI=y
CONFIG_SECURITY_NETWORK=y
EXAMPLE B: BluetoothProxy on wench
----------------------------------------------
wench:~# egrep "XEN|PCI|USB|HCI|BT|VID"
/boot/config-2.6.11.12-BluetoothProxy | grep -v "#"
CONFIG_XEN=y
CONFIG_ARCH_XEN=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_WRITABLE_PAGETABLES=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XENARCH="i386"
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_LEGACY_PROC=y
CONFIG_PCI_NAMES=y
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_BT=m
CONFIG_BT_L2CAP=m
CONFIG_BT_SCO=m
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_HIDP=m
CONFIG_BT_HCIUSB=m
CONFIG_BT_HCIUSB_SCO=y
CONFIG_BT_HCIVHCI=m
CONFIG_VIDEO_DEV=m
CONFIG_USB=m
CONFIG_USB_DEVICEFS=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_UHCI_HCD=m
CONFIG_USB_STV680=m
====================================================================
3. Configuration of driver domain
wench:~# grep = /etc/xen/BluetoothProxy
#============================================================================
kernel = "/boot/vmlinuz-2.6.11.12-BluetoothProxy"
pci = ['00,7,2']
memory = 80
name = "BluetoothProxy"
disk = [ 'phy:/dev/vg0/BluetoothProxy,/dev/hda3,w']
root = "/dev/hda3 ro"
extra = "4"
restart = 'onreboot'
#============================================================================
====================================================================
4. Confirming results
EXAMPLE A: firewall on brutus
---------------------------------------------
tba
EXAMPLE B: BluetoothProxy on wench
----------------------------------------------------------
BluetoothProxy:~# lspci
0000:00:04.2 USB Controller: Intel Corp. 82371AB/EB/MB PIIX4 USB (rev 01)
(Yep, that's ALL it comes up with!:))
BluetoothProxy:~# lsmod
Module Size Used by
stv680 26060 0
ipt_MASQUERADE 2624 0
iptable_nat 19836 1 ipt_MASQUERADE
ip_conntrack 39224 2 ipt_MASQUERADE,iptable_nat
ip_tables 19904 2 ipt_MASQUERADE,iptable_nat
rfcomm 35132 0
bnep 12256 2
crc32 3808 1 bnep
l2cap 22340 9 rfcomm,bnep
videodev 7360 1 stv680
hci_usb 13928 2
bluetooth 44164 8 rfcomm,bnep,l2cap,hci_usb
uhci_hcd 30416 0
usbcore 107672 4 stv680,hci_usb,uhci_hcd
BluetoothProxy:~# lsusb
Bus 001 Device 004: ID 0553:0202 STMicroelectronics Imaging Division
(VLSI Vision) Aiptek PenCam 1
Bus 001 Device 003: ID 0db0:1967 Micro Star International Bluetooth Dongle
Bus 001 Device 001: ID 0000:0000
Using iPaq to connect to BluetoothProxy NAP service, we see:
BluetoothProxy:~# tail /var/log/syslog
Aug 2 20:31:08 localhost hcid[1739]: link_key_request
(sba=00:10:DC:E9:71:7C, dba=00:04:3E:A3:C7:73)
Aug 2 20:31:08 localhost pand[9838]: New connection from
00:04:3E:A3:C7:73 bnep0
Testing the camera with pencam2 (must remember to null out stv680 in
modules)
This is the look you give when it all just works:
http://marcusbrutus.cust.internode.on.net/Computers/Me/snapshot002
====================================================================
5. References
5.1 Driver Domain Configuration
The Xen Users' Manual, chapter 7.3
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html#SECTION03230000000000000000
Flat Format: Include a list of PCI device addresses of the format:
pci = ['x,y,z', ...]
where each element in the list is a string specifying the components of
the PCI device address, separated by commas. The components (x, y and z)
of the list may be formatted as either decimal or hexadecimal.
*****************************************************************************************************************
Andreas Seuss wrote:
> Hi Marcus,
>
> I read your mail on the Xen Users mailing list. What I try to achieve
> is to share a pci network interface card. I compiled a kernel
> according to your config shown below except that I used the NIC
> modules. I run xen-3.0 (devel). When I include the pci = ['00,08,00']
> statement into my config file I get the following error when trying to
> create the domain:
>
> Error: Error creating domain: global name 'bus' is not defined
>
> I also set the 'physdev_dom0_hide=(00:08.0)' in the menu list. Tried
> it on the kernel line and on the module line (not both on the same
> time;-))
> but it does not seem to work. I can still see all pci devices. Maybe
> xou could paste the respective line from your menu.lst?
The hide stanza goes in the kernel line.
Can you confirm the PCI ID with lspci in dom0?
>
> Which documentation did you use? The user manual? It contains only
> sparse information on that topic I think. Thanks in advance.
Yes, the user manual :) (see References above).
Not as verbose as my attempt, huh? :)
> Andreas.
>
>
> Marcus Brown wrote:
>
>> Heh,
>> ipaq turned off, bluez-utils not installed, no IRQ for USB... etc
>> sorry to trouble the list :) ... thanks for reading!!
>>
>> ie. PCI + USB working afaik (not fully tested but looks ok)
>>
>> M.
>>
>> Marcus Brown wrote:
>>
>>
>>
>>> OK, PCI export working now.
>>>
>>> Except that I still can't configure the bnep0 interface.
>>> Hot plugging seems to work, but dmesg gives:
>>> SIOCSIFADDR: No such device
>>> bnep0: ERROR while getting interface flags: No such device
>>> SIOCSIFNETMASK: No such device
>>> bnep0: ERROR while getting interface flags: No such device
>>> Failed to bring up bnep0.
>>>
>>> # lspci
>>> 0000:00:07.2 USB Controller: Intel Corp. 82371AB/EB/MB PIIX4 USB
>>> (rev 01)
>>> # lsusb
>>> Bus 001 Device 002: ID 0db0:1967 Micro Star International Bluetooth
>>> Dongle
>>> Bus 001 Device 001: ID 0000:0000
>>> # lsmod
>>> Module Size Used by
>>> bnep 12256 0
>>> crc32 3808 1 bnep
>>> l2cap 22340 1 bnep
>>> videodev 7360 0
>>> hci_usb 13928 0
>>> bluetooth 44164 3 bnep,l2cap,hci_usb
>>> uhci_hcd 30416 0
>>> usbcore 107672 3 hci_usb,uhci_hcd
>>>
>>> Any suggestions?
>>>
>>> (Yes, I talk to myself in the blue room too !! :))
>>>
>>> Marcus Brown wrote:
>>>
>>>
>>>
>>>
>>>
>>>> Dear, oh dear!!
>>>>
>>>> pci = ['00,07,2'] in domU config!!!
>>>> (I'm going blind, I think)
>>>>
>>>> doh!
>>>>
>>>> lol
>>>>
>>>> Marcus Brown wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Hi all.
>>>>>
>>>>> I've been attempting to export the USB Host Controller to a domU,
>>>>> so that I can use a USB Bluetooth adapter (and possibly a USB
>>>>> camera).
>>>>> I've been using xen-testing on Debian (dom0 is sid, domU is sarge)
>>>>>
>>>>> I've managed to get the domU to boot using a few of the following
>>>>> settings:
>>>>>
>>>>> The kernel line in grub's menu.lst now includes:
>>>>> physdev_dom0_hide=(00:07.2)
>>>>> The domU kernel config has:
>>>>> XEN --->
>>>>> [*] Physical device access
>>>>> [*] Block-device backend driver
>>>>> [ ] Network-device backend driver
>>>>> X86 Processor Configuration --->
>>>>> Bus options (PCI, PCMCIA, EISA, MCA, ISA) --->
>>>>> [*] PCI support
>>>>> [*] Legacy /proc/pci interface
>>>>> [*] PCI device name database
>>>>> Device Drivers --->
>>>>> Networking support --->
>>>>> <M> Bluetooth subsystem support --->
>>>>> --- Bluetooth subsystem support
>>>>> <M> L2CAP protocol support
>>>>> <M> SCO links support
>>>>> <M> RFCOMM protocol support
>>>>> [*] RFCOMM TTY support
>>>>> <M> BNEP protocol support
>>>>> [*] Multicast filter support
>>>>> [*] Protocol filter support
>>>>> <M> HIDP protocol support
>>>>> Bluetooth device drivers --->
>>>>> <M> HCI USB driver
>>>>> [*] SCO (voice) support
>>>>> <M> HCI VHCI (Virtual HCI device) driver
>>>>> Device Drivers --->
>>>>> Multimedia devices --->
>>>>> <M> Video For Linux
>>>>> USB support --->
>>>>> <M> Support for Host-side USB
>>>>> [*] USB device filesystem
>>>>> <M> UHCI HCD (most Intel and VIA) support
>>>>> <M> USB STV680 (Pencam) Camera support
>>>>>
>>>>> After booting the domU, the USB modules can be loaded manually,
>>>>> however, there are no PCI devices found, or USB devices listed.
>>>>>
>>>>> I have come to the conclusion (from the xen-users manual) that my
>>>>> xend-config.sxp is in error. I've tried appending:
>>>>> (backend (blkif))
>>>>> (device (pci (bus 00) (dev 07) (func 2)))
>>>>>
>>>>> (this could explain why I now get errors from 'xm list -v' now:))
>>>>>
>>>>> but during boot of the domU I still get the following:
>>>>> Starting hotplug subsystem:
>>>>> pci
>>>>> cat: /sys/bus/pci/devices/*/class: No such file or directory
>>>>> cat: /sys/bus/pci/devices/*/vendor: No such file or directory
>>>>> cat: /sys/bus/pci/devices/*/device: No such file or directory
>>>>> cat: /sys/bus/pci/devices/*/subsystem_vendor: No such file or
>>>>> directory
>>>>> cat: /sys/bus/pci/devices/*/subsystem_device: No such file or
>>>>> directory
>>>>> Bad PCI agent invocation
>>>>>
>>>>> Can anyone see something obviously wrong here?
>>>>> The users manual talks about placing these options
>>>>> "somewhere within the vm element of its configuration"
>>>>> but I'm unable to find information on how to do this.
>>>>>
>>>>> Thanks for any pointers,
>>>>>
>>>>> Marcus.
>>>>>
>>>>> _______________________________________________
>>>>> Xen-users mailing list
>>>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>>>> http://lists.xensource.com/xen-users
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-users mailing list
>>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-users
>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-users
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>>
>>
>
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home