Re: [Xen-users] Re: VT-D RMRR is incorrect

[Yoshiharu Mori <ymori4480@xxxxxxxxx>]

Hi Christian

I checked the following spcifications.

 "Intel Virtualization Technology (Intel VT) for Directed I/O Architecture Specification"
 address=> http://download.intel.com/technology/computing/vptech/Intel(r)_VT_for_Direct_IO.pdf

As a result, DMAR ACPI TABLE of SUPERMICRO C7X58's BIOS violates the specification.

Here is the DMAR ACPI table: (I use acpidump command)

DMAR @ 0xbf79e0c0
  0000: 44 4d 41 52 18 01 00 00 01 92 41 4d 49 00 00 00  DMAR......AMI...
  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
  0020: 97 00 00 00 26 01 00 00 00 00 00 00 00 00 00 00  ....&...........
  0030: 00 00 18 00 01 00 00 00 00 e0 ff fb 00 00 00 00  ................
  0040: 03 08 00 00 08 f0 1f 07 01 00 58 00 00 00 00 00  ..........X.....
  0050: 00 c0 0e 00 00 00 00 00 ff ff 0e 00 00 00 00 00  ................
  0060: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
  0070: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
  0080: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
  0090: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................
  00a0: 01 00 58 00 00 00 00 00 00 c0 7e bf 00 00 00 00  ..X.......~.....
  00b0: ff bf 7e bf 00 00 00 00 01 08 00 00 00 00 1d 00  ..~.............
  00c0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  00d0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  00e0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  00f0: 01 08 00 00 00 00 1a 07 02 00 20 00 00 00 00 00  .......... .....
  0100: 02 08 00 00 00 00 01 00 02 08 00 00 00 00 03 00  ................
  0110: 02 08 00 00 00 00 07 00                          ........

There are two Reserved Memory Region Reporting (RMRR) Structure in above DMAR ACPI TABLE.
The 1st RMRR is correct( I think) , But the 2nd RMRR is incorrect.
(Please look at page 75 of Intel's specifications.)

The RMRR  Limit Address must be greater than the RMRR Base Adress.
But, in my BIOS, RMRR Base Address is bf7ec000 and RMRR Limit Address is bf7ebfff.
(RMRR Limit Address is smaller than RMRR Base Address !)

Your ASUS P6T mother board and my board uses AMI BIOS.
All AMI BIOS might be wrong in how to handle RMRR.

I sent this report to SUPERMICRO Technical team.

Thanks

Yoshiharu Mori

2009/2/15 Christian Tramnitz <chris.ace@xxxxxxx>

Hello Yoshiharu-san!

Have you tried 3.4-unstable yet?
I had the same problem with my Asus P6T Deluxe board (also X58) and got a little further on 3.4 but finally (after some more RMRR and DMAR details displayed) the same error message.

It would be interesting to know if just the BIOSes (their DMAR maps to be more specific) on non-Intel manufactured X58 boards are broken or if there are general problems with X58 and Xen... has anyone gotten VTD/IOMMU to work on X58 yet?

Best regards,
  Christian

Yoshiharu Mori schrieb:

I try pci pci passthrough with xen 3.3.1 and CentOS 5.2(64bit)
on a SUPERMICRO C7X58 board

I see the following the error in my boot log.

(XEN) [VT-D]dmar.c:372: RMRR is incorrect.

This problem is caused by this condition in dmr.c:372.

 if ( rmrr->base_address >= rmrr->end_address )
 {
  dprintk(XENLOG_ERR VTDPREFIX, "RMRR is incorrect.\n");
  return -EFAULT;
 }

As an experiment, I delete the "return -EFAULT" and re-compiling Xen,
I can pass a PCI device through to Gest-OS(Windows XP) and have no problem
with using the devices on Gest-OS.

Is the above condition really correct and important ?

xm dmesg results with deleting "return -EFAULT" are following.

[root@localhost ~]# xm dmesg
 __  __            _____  _____  _
 \ \/ /___ _ __   |___ / |___ / / |
 \  // _ \ '_ \    |_ \   |_ \ | |
 /  \  __/ | | |  ___) | ___) || |
 /_/\_\___|_| |_| |____(_)____(_)_|

(XEN) Xen version 3.3.1 (root@(none)) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) Sat Feb 14 05:21:09 JST 2009
(XEN) Latest ChangeSet: unavailable
(XEN) Command line: iommu msi=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000099400 (usable)
(XEN)  0000000000099400 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000bf790000 (usable)
(XEN)  00000000bf790000 - 00000000bf79e000 (ACPI data)
(XEN)  00000000bf79e000 - 00000000bf7d0000 (ACPI NVS)
(XEN)  00000000bf7d0000 - 00000000bf7e0000 (reserved)
(XEN)  00000000bf7ec000 - 00000000c0000000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ffc00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 00000001c0000000 (usable)
(XEN) System RAM: 6135MB (6282404kB)
(XEN) ACPI: RSDP 000F9C50, 0024 (r2 ACPIAM)
(XEN) ACPI: XSDT BF790100, 0054 (r1 112608 XSDT1738 20081126 MSFT       97)
(XEN) ACPI: FACP BF790290, 00F4 (r4 112608 FACP1738 20081126 MSFT       97)
(XEN) ACPI: DSDT BF790470, 5AA5 (r2  1F380 1F380000        0 INTL 20051117)
(XEN) ACPI: FACS BF79E000, 0040
(XEN) ACPI: APIC BF790390, 0092 (r2 112608 APIC1738 20081126 MSFT       97)
(XEN) ACPI: MCFG BF790430, 003C (r1 112608 OEMMCFG  20081126 MSFT       97)
(XEN) ACPI: OEMB BF79E040, 0072 (r1 112608 OEMB1738 20081126 MSFT       97)
(XEN) ACPI: DMAR BF79E0C0, 0118 (r1    AMI  OEMDMAR        1 MSFT       97)
(XEN) ACPI: SSDT BF7A0260, 1298 (r1 DpgPmm    CpuPm       12 INTL 20051117)
(XEN) Xen heap: 14MB (14536kB)
(XEN) Domain heap initialised
(XEN) Processor #0 7:10 APIC version 21
(XEN) Processor #2 7:10 APIC version 21
(XEN) Processor #4 7:10 APIC version 21
(XEN) Processor #6 7:10 APIC version 21
(XEN) Processor #1 7:10 APIC version 21
(XEN) Processor #3 7:10 APIC version 21
(XEN) Processor #5 7:10 APIC version 21
(XEN) Processor #7 7:10 APIC version 21
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) [VT-D]dmar.c:372: RMRR is incorrect.
(XEN) Intel VT-d has been enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2660.058 MHz processor.
(XEN) VMX: EPT is available.
(XEN) VMX: VPID is available.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) CPU0: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 1/2 eip 8c000
(XEN) CPU1: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 2/4 eip 8c000
(XEN) CPU2: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 3/6 eip 8c000
(XEN) CPU3: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 4/1 eip 8c000
(XEN) CPU4: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 5/3 eip 8c000
(XEN) CPU5: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 6/5 eip 8c000
(XEN) CPU6: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Booting processor 7/7 eip 8c000
(XEN) CPU7: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 04
(XEN) Total of 8 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) checking TSC synchronization across 8 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 8 CPUs
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) [VT-D]iommu.c:1269:d32767 domain_context_mapping:invalid
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 0xffffffff805c190c
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   00000001b6000000->00000001b8000000 (1498227 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80200000->ffffffff805c190c
(XEN)  Init. ramdisk: ffffffff805c2000->ffffffff80b88200
(XEN)  Phys-Mach map: ffffffff80b89000->ffffffff81707398
(XEN)  Start info:    ffffffff81708000->ffffffff817084a4
(XEN)  Page tables:   ffffffff81709000->ffffffff81718000
(XEN)  Boot stack:    ffffffff81718000->ffffffff81719000
(XEN)  TOTAL:         ffffffff80000000->ffffffff81800000
(XEN)  ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 104kB init memory.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users