[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Mapping HVM guest memory from Dom0


  • To: Spencer Michaels <spmichaels.work@xxxxxxxxx>, <tamas.k.lengyel@xxxxxxxxx>
  • From: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • Date: Thu, 11 Oct 2018 00:10:37 +0100
  • Autocrypt: addr=andrew.cooper3@xxxxxxxxxx; prefer-encrypt=mutual; keydata= xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs 6+ahAA==
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx
  • Delivery-date: Wed, 10 Oct 2018 23:10:55 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 10/10/18 23:08, Spencer Michaels wrote:
> Interesting … sorry, I had read the docs a while ago and my
> interpretation at the time was that it didn't. I can try to get libvmi
> working, but nonetheless I do want to figure out how to this with the
> Xen API itself if at all possible, so I'd appreciate any help in doing
> so.
>
> I've looked through libvmi's implementation some more; it looks like
> it does this via vmi_pagetable_lookup_cache, which in my case ends up
> calling v2p_pae (intel.c:327). As I understand it, in that function,
> the `dtb` parameter holds the pagetable base address, and you use that
> to walk the page table and get the physical address corresponding to
> the virtual addr. Based on the end of vmi_translate_kv2p
> (accessors.c:703), it looks like the value of dtb is `vmi->kpgd`,
> which at some point earlier is set to the value of the `cr3` register.
> Is my understanding, roughly speaking, correct?
>
> I tried to replicate a simpler version of libvmi's page table walking
> code just now, and in doing so noticed that the value of CR0, as
> reported by Xen, has its 31st bit set to zero … i.e. virtual
> addressing is disabled entirely? (For reference, I'm using a 64-bit
> Ubuntu image set up initially by virt-manager but now just run via `xl
> create`. I don't know if I should expect it to have paging enabled or
> not.) On the other hand, if I understand correctly, what I mentioned
> earlier with regards to some HVM guests assuming (addr >>
> XC_PAGE_SHIFT) = MFN *is* assuming that guest page frame number =
> machine frame number, which sounds like it is what would be applicable
> in this case. 
>
> Honestly, I'm not so sure what's happening here, and I'm not even sure
> my description is sensible at this point — I need to look into
> libvmi's implementation more and also figure out exactly what paging
> mode (or lack thereof) my HVM guest is running in. If, on the other
> hand, any of the above sounds familiar, suggestions/hints would be
> very helpful.

Here are some observations/points which may help.

x86 is a bit more complicated than other architectures, and as a result,
there is a lot of subtly incorrect terminology (including in Libvmi -
sorry Tamas).

A virtual address (also called an effective address) is a segment:offset
pair.  The segment is almost always implicit (%cs for instruction
fetches, %ss for stack accesses, %ds for normal data accesses).  The
segmentation part of address translation adds the segment base to the
offset to produce a linear address.

A "flat memory model" (used by almost all 32bit OSes and is
unconditional in AMD64) is one where the segment base is 0, at which
point offset == linear address.  This is where most confusion over the
term "virtual address" arises.

The paging part of address translation takes a linear address, follows
the pagetable structure (rooted in %cr3), to produce a physical address
as an output.  This may be guest physical or host physical depending on
the VM configuration.


Next, please read
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/include/xen/mm.h;h=054d02e6c0e68b411afba42424dc5fe7e7d69855;hb=refs/heads/staging#l8
which describes the terminology Xen uses for various address spaces.  In
particular, the difference between MFN and GFN.

The relevant difference PV and HVM guests as far as you are concerned is
that for PV guests, GFN == MFN because the pagetables written by the
guest are walked directly by hardware.

An HVM guest has GFN != MFN, because the guest physical to host physical
translation is provided by HAP/EPT/NPT/SLAT (whichever term you chose to
use for hardware acceleration), or by the shadow pagetables (maintained
and operated by Xen, for hardware lacking HAP support).

The foreign map API uses a GFN, even if the underlying API describes the
parameter name as MFN.  This is a consequence of PV guests having been
developed long before hardware virt extensions came along, and noone
having gone through and retroactively updated the terminology.

Therefore, for both PV and HVM guests, you can take the guest cr3,
extract the frame part of it, and ask the foreign map API to map that
guest frame.  The underlying implementation in Xen is trivial for a PV
guest (GFN == MFN), but slightly more complicated for HVM guests (GFN
has to be translated into an MFN by Xen walking the HAP/shadow
pagetables before dom0's mapping request can be completed).

I hope this clears up some of the confusion.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.