Re: Re: Memory corruption bug with Xen PV Dom0 and BOSS-S1 RAID card

[Paweł Srokosz <pawel.srokosz@xxxxxxx>]

Hello,

> So the issue doesn't happen on debug=y builds? That's unexpected.  I would
> expect the opposite, that some code in Linux assumes that pfn + 1 == mfn +
> 1, and hence breaks when the relation is reversed.

It was also surprising for me but I think the key thing is that debug=y
causes whole mapping to be reversed so each PFN lands on completely different
MFN e.g. MFN=0x1300000 is mapped to PFN=0x20e50c in ndebug, but in debug
it's mapped to PFN=0x5FFFFF. I guess that's why I can't reproduce the
problem.

> Can you see if you can reproduce with dom0-iommu=strict in the Xen command
> line?

Unfortunately, it doesn't help. But I have few more observations.

Firstly, I checked the "xen-mfndump dump-m2p" output and found that misread
blocks are mapped to suspiciously round MFNs. I have different versions of
Xen and Linux kernel on each machine and I see some coincidence.

I'm writing few huge files without Xen to ensure that they have been written
correctly (because under Xen both read and writeback is affected). Then I'm
booting to Xen, memory-mapping the files and reading each page. I see that when 
block is corrupted, it is mapped on round MFN e.g. pfn=0x5095d9/mfn=0x1600000, 
another on pfn=0x4095d9/mfn=0x1500000 etc.

On another machine with different Linux/Xen version these faults appear on
pfn=0x20e50c/mfn=0x1300000, pfn=0x30e50c/mfn=0x1400000 etc.

I also noticed that during read of page that is mapped to
pfn=0x20e50c/mfn=0x1300000, I'm getting these faults from DMAR:

```
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 1200000000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 1200001000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 1200006000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 1200008000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 1200009000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 120000a000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:65:00.0] fault addr 120000c000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
```

and every time I'm dropping the cache and reading this region, I'm getting
DMAR faults on few random addresses from 1200000000-120000f000 range (I guess 
MFNs 0x1200000-120000f). MFNs 0x1200000-0x12000ff are not mapped to any PFN in
Dom0 (based on xen-mfndump output.). 

On the other hand, I'm not getting these DMAR faults while reading other 
regions.
Also I can't trigger the bug with reversed Dom0 mapping, even if I fill the page
cache with reads.

Thank you,
Paweł