Hi George,
I found that you have submitted a patch about “libxl,hvmloader: Don't relocate memory for MMIO hole”. So, I have a “8G large bar PCI device(NVIDIA Tesla M60)” passthrough question to ask you.
Host passthrough PCI Device info:
lspci -vs 06:00.0
06:00.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
Subsystem: NVIDIA Corporation Device 115e
Flags: fast devsel, IRQ 40
Memory at 93000000 (32-bit, non-prefetchable) [disabled] [size=16M]
Memory at 23c00000000 (64-bit, prefetchable) [disabled] [size=8G]
Memory at 23e00000000 (64-bit, prefetchable) [disabled] [size=32M]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [250] Latency Tolerance Reporting
Capabilities: [258] #1e
Capabilities: [128] Power Budgeting <?>
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [900] #19
Kernel driver in use: pciback
Kernel modules: nvidiafb
Guest passthrough PCI Device info:
lspci -vs 00:05.0
00:05.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
Subsystem: NVIDIA Corporation Device 115e
Physical Slot: 5
Flags: fast devsel, IRQ 36
Memory at f5000000 (32-bit, non-prefetchable) [size=16M]
Memory at 200000000 (64-bit, prefetchable) [size=4G]
Memory at f2000000 (64-bit, prefetchable) [size=32M]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Kernel modules: nouveau, nvidiafb
passthrough to the guest, the large bar size only has 4G. so, the passthrough pci device does not work, windows 7/2008 R2 64bit vm has the same problem.
.
According to the analysis hvmloader code, find a problem:
if (is_64bar) {
bar_data_upper = pci_readl(devfn, bar_reg + 4);
pci_writel(devfn, bar_reg + 4, ~0);
bar_sz_upper = pci_readl(devfn, bar_reg + 4);
pci_writel(devfn, bar_reg + 4, bar_data_upper);
bar_sz = (bar_sz_upper << 32) | bar_sz;
}
bar_sz &= ~(bar_sz - 1);
read from the pci device, bar_sz_upper is 0xffffffff, if the bar size is 8G, the bar_sz_upper should be 0xfffffffe.
Have you ever encountered a similar problem?
Env info:
device_model_version: qemu-xen
xen version: xen-4.6.1
Guest: RedHat-6.4-64
Passthrough PCI Device: NVIDIA Tesla M60(compute mode)
CPU: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
Thanks
Feng
