|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [PATCH v6 00/44] ASI: HVM through X4
This is the first part of a series to implement address space isolation. The approach is the same as v3 to v5: Start by focusing on just getting ASI and intra-domain isolation for HVM guests. PV support is added where it's simple to do. The full series can be roughly divided as below: Included in this version, all posted before: - VH (01-13): Give each vCPU of an HVM guest its own per-domain area and its own mapcache. - DV (14-23): Maintain two "views" of the directmap: A full directmap, and a sparse directmap. The full directmap is what we have now; the sparse directmap removes non-xenheap heap memory. - X1 (24-30): Add a vCPU state window to the per-domain area (both for PV and HVM). Move SVM VMCB structures and VMX guest MSR areas into it. - X2 (31-34): Move every vCPU's saved register frame (PV and HVM), and the HVM emulator's operand caches, into the vCPU state window. 34 also clears the vCPU's copy of an emulated I/O request once it has been consumed. - X3 (35-37): Introduce map_domain_pages(), which maps several pages in the calling vCPU's per-domain area, and use it in place of vmap() for the shadow code's cross-page emulated writes and the HVM emulator's multi-frame accesses. - X4 (38-44): Move the guest XSAVE areas into the vCPU state window. Implemented but not yet posted: - PH1: Preparation for per-pCPU stacks, modifying rendezvous and call-function IPIs to not use the stack - PH2: Moving pCPU stacks into the perdomain area, allocating them from the domheap - O: A number of optimizations to improve performance The functionality of v6 (through X4) is the bare minimum that XenServer needs to make it into 4.23. v5 can be found here: https://lore.kernel.org/xen-devel/20261028-asi-hvm-v5-0-13b908e0df0b@xxxxxxxxxxxxxx/ v6 is based on staging 2043d22b7e92. v5's 01, "x86/PoD: map one page at a time in p2m_pod_zero_check()", is now in staging (41ddcfe808ec), so v5's NN is v6's NN-1. The changes since v5 have been tested (below) and skimmed, but not yet been over with a fine-tooth comb. Changes in v6: - Following Jan's suggestion on v5's 02, the per-domain L3 is now allocated when the domain is created, instead of refusing to promote an L4 while there is none: 01, "x86/mm: allocate the per-domain L3 in arch_domain_create()", replaces v5's "x86/mm: refuse to promote an L4 while the domain has no per-domain L3". A vCPU-PT domain has no domain-wide L3: each of its vCPUs gets its own in arch_vcpu_create() (08). create_perdomain_mapping() no longer allocates an L3. - To go with it, 07 gives init_xen_l4_slots() two forms around a common core: one taking the vCPU that a root page table is private to (HVM monitor tables, the compat L4), and init_xen_l4_slots_pv(), taking only the domain, for the root page tables a PV domain's vCPUs share (its L4s and their shadows). promote_l4_table() therefore no longer needs vCPU 0. - 02, 04, 05, 20 and 25 change to match, and 13 now cites the PoD fix by its hash; each patch's changelog has the details. Jan's Reviewed-by on v5's 06 (now 05) has been dropped. - In 16, sparse_dmap_init() is still called once the heap has all memory. Since "x86: invoke xenheap_max_mfn() unconditionally" (f93ffa1309cd), that means after the loop for RAM above 5 TiB, rather than right after the change to SYS_STATE_boot. Some more details: 15 is a work-around for a slow-down which causes a watchdog to fire. (Proper performance fixes have been implemented but won't be posted until the end of the entire series.) 34 also limits exposure of ioreq state: Completely isolating each vCPU's I/O requests from being read by other vCPUs of the same domain is complicated; as a mitigation, this patch clears the emulated I/O request once it has been consumed, reducing the window available to read the payload. 38-41 are Alejandro's patches from "x86: Address Space Isolation FPU preparations" (v3, January 2025), revived. State is moved from the xenheap into the per-domain area for both ASI and non-ASI domains: VMCB, MSR List, vCPU register frame, and XSAVE/FPU areas. Same is true for users of map_domain_pages. This protects non-ASI domains from ASI domains. One key thing to get feedback on is the question below 44's "---": xstate_init() now panics on hardware whose XSAVE area needs more than the three pages reserved for it in the vCPU state window. No current hardware does. Should it do more, for instance size the reservation from the size found at boot, or fall back to xenheap areas on such hardware? Note that at the end of this series, PV guests can be given the sparse directmap; this isolates them from other domains, but doesn't isolate vCPUs within a domain. With this series, a PV domain's register frames and XSAVE areas are also out of reach of domains on the sparse directmap, whether or not the PV domain itself is on it. Testing of v6: - On the series as reworked, on staging 8b0a70ddd29d, before the rebase (so without "x86: invoke xenheap_max_mfn() unconditionally" and without the smaller review fixes): - Builds of xen/ at each of the 44 patches, in seven configurations: debug and release, !HVM, !PV, the PV shim, and shadow paging, debug and release. No warnings or errors beyond those staging itself gives. - On an Intel NUC: whole-tree builds (xen, tools, stubdom, docs) at each of the 44, with tools/tests/vpci run. - QEMU (TCG) smoke boots at 08 and 44: PV and PVH dom0s (the PVH one on HAP and on shadow), and XTF's 32-bit PV and 32-bit HVM selftests as dom0, without asi= and with asi=1. Each boot gets to dom0's userspace (or the test's result) with no crash, assertion or warning in Xen's log. - On the NUC: boots of 08 and 44 without asi= and with asi=1. In each, a 64-bit PV guest is live-migrated to the same host in shadow log-dirty mode under a verified workload, with an HVM guest beside it; with asi=1, XTF's full suite, 32-bit PV included, passes twice (the usual four skips). - On the rebased series: each patch's tree differs from the tested one only by staging's changes and the review fixes in 01, 07, 08 and 16, checked file by file. The same builds and boots are being repeated on it; those finished so far have passed. - The same basic code has been run through XenRT, with the issue addressed by 15 (slow dom0 kernel decompression triggering a watchdog on large machines) the only issue found. A dependency graph of the above functionality can be found here: https://xenbits.xenproject.org/people/gdunlap/asi-series-deps.html Extending the intra-domain isolation to PV guests would involve implementing per-vCPU root pagetables for PV guests. There are draft implementations of this, but they are quite complicated. Alejandro Vallejo (4): x86/emulator: Refactor FXSAVE_AREA to use wrappers x86/mpx: Map/unmap xsave area in read_bndcfgu() x86/fpu: Pass explicit xsave areas to fpu_(f)xsave() x86/fpu: Pass explicit xsave areas to fpu_(f)xrstor() George Dunlap (30): x86/mm: allocate the per-domain L3 in arch_domain_create() x86/mm: have init_xen_l4_slots() take a vCPU in place of the domain x86/domain_page: drop redundant is_pv_vcpu() checks x86/domain_page: don't unmap a mapcache entry that is not present x86/mm: let Xen's page-table walker operate on a given root xen/gunzip: process pending softirqs once per deflate block x86/mm: add hooks for a sparse view of the directmap x86/mm: build and maintain a sparse view of the directmap x86/domain_page: key the directmap fast paths on the loaded view x86/domain_page: clear a mapcache slot at its last unmap x86/mm: run guest contexts on the sparse directmap view x86/spec-ctrl: wire the sparse directmap view into the asi= option xen/xmalloc: return an emptied region after dropping the pool lock x86/mm: split a sparse directmap superpage with a page being freed x86/mm: introduce a per-vCPU state window in the perdomain area x86/svm: introduce SVM_MAP_VMCB() and SVM_UNMAP_VMCB() x86/svm: restore guest debug registers in svm_do_resume() x86/svm: drop svm_vmcb_dump()'s paddr x86/svm: allocate VMCBs from the domain heap behind the vCPU state window x86/vmx: move the guest MSR load/save list off always-mapped memory x86: access a vCPU's saved register frame through map/unmap helpers x86: move the saved register frame behind the vCPU state window x86/hvm: move the emulator's operand caches behind the vCPU state window x86/hvm: clear the vCPU's copy of an emulated I/O request once consumed x86/domain_page: introduce map_domain_pages() x86/shadow: use map_domain_pages() for cross-page emulated writes x86/hvm: use map_domain_pages() for multi-frame emulated accesses x86/xstate: classify xsave-area seam accesses by failure mode x86/domain: restore FPU state after switching to the new page tables x86/xstate: move guest XSAVE areas behind the vCPU state window Roger Pau Monné (10): x86/mm: purge unneeded destroy_perdomain_mapping() x86/mm: prepare destroy_perdomain_mapping() for per-vCPU perdomain areas x86/domain_page: drop redundant create_perdomain_mapping() call x86/mm: prepare create_perdomain_mapping() for per-vCPU perdomain areas x86/spec-ctrl: introduce Address Space Isolation command line option x86/hvm: introduce per-vCPU L3 page-table x86/domain_page: prepare the mapcache for per-vCPU accounting x86/domain_page: move struct mapcache_vcpu to struct arch_vcpu x86/hvm: introduce per-vCPU mapcache for vCPU-PT domains x86/mm: introduce populate_perdomain_mapping() docs/misc/xen-command-line.pandoc | 57 +++ xen/arch/x86/Kconfig | 1 + xen/arch/x86/Makefile | 1 + xen/arch/x86/domain.c | 239 ++++++++-- xen/arch/x86/domain_page.c | 461 ++++++++++++++++---- xen/arch/x86/domctl.c | 98 +++-- xen/arch/x86/hvm/domain.c | 6 +- xen/arch/x86/hvm/emulate.c | 243 ++++++++--- xen/arch/x86/hvm/hvm.c | 122 +++--- xen/arch/x86/hvm/svm/nestedsvm.c | 12 +- xen/arch/x86/hvm/svm/svm.c | 147 +++++-- xen/arch/x86/hvm/svm/vmcb.c | 74 +++- xen/arch/x86/hvm/svm/vmcb.h | 29 +- xen/arch/x86/hvm/vmx/vmcs.c | 151 ++++++- xen/arch/x86/hvm/vmx/vmx.c | 7 +- xen/arch/x86/hypercall.c | 4 +- xen/arch/x86/i387.c | 60 ++- xen/arch/x86/include/asm/config.h | 42 +- xen/arch/x86/include/asm/domain.h | 109 ++++- xen/arch/x86/include/asm/hvm/emulate.h | 9 +- xen/arch/x86/include/asm/hvm/vcpu.h | 7 + xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 63 +-- xen/arch/x86/include/asm/i387.h | 1 + xen/arch/x86/include/asm/mm.h | 32 +- xen/arch/x86/include/asm/sparse-directmap.h | 83 ++++ xen/arch/x86/include/asm/spec_ctrl.h | 3 + xen/arch/x86/include/asm/xstate.h | 73 ++-- xen/arch/x86/mm.c | 459 ++++++++++++++++--- xen/arch/x86/mm/hap/hap.c | 2 +- xen/arch/x86/mm/shadow/hvm.c | 6 +- xen/arch/x86/mm/shadow/multi.c | 5 +- xen/arch/x86/pv/dom0_build.c | 11 +- xen/arch/x86/pv/domain.c | 30 +- xen/arch/x86/pv/shim.c | 4 +- xen/arch/x86/setup.c | 4 + xen/arch/x86/sparse-directmap.c | 455 +++++++++++++++++++ xen/arch/x86/spec_ctrl.c | 183 +++++++- xen/arch/x86/traps.c | 20 +- xen/arch/x86/x86_64/mm.c | 14 +- xen/arch/x86/x86_emulate/blk.c | 36 +- xen/arch/x86/xstate.c | 212 ++++++++- xen/common/Kconfig | 18 + xen/common/gzip/gunzip.c | 1 + xen/common/gzip/inflate.c | 1 + xen/common/page_alloc.c | 51 ++- xen/common/xmalloc_tlsf.c | 12 +- xen/include/xen/domain_page.h | 21 + xen/include/xen/mm.h | 40 ++ 48 files changed, 3101 insertions(+), 618 deletions(-) create mode 100644 xen/arch/x86/include/asm/sparse-directmap.h create mode 100644 xen/arch/x86/sparse-directmap.c base-commit: 2043d22b7e926f508cc23391b4c75325fa7211e9 -- 2.56.0
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |