[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 2/2] x86/xen: Allow per-domain usage of hardware virtualized APIC


  • To: Jane Malalane <Jane.Malalane@xxxxxxxxxx>
  • From: Christian Lindig <christian.lindig@xxxxxxxxxx>
  • Date: Thu, 27 Jan 2022 16:21:12 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6s2fVJGJBPtumMsP5lLEO2XTE2qiMqZ42jvqtZRE85U=; b=Cs2p6O0iRXawvTuzvedbykeK2MftCKV3G/x0vW+3amV0ZOnN9UVL+hokeJS0NzyLNYklyOsYI9aaDJ24ZlIK6y/id1Qxj2a7jzcMnXX/PQD20Ayrc2AFYCDfdkv0gBx+ZthmPlE/DKDiW5nzrm5VYlN8jWz12GozuOVDMewaKMNB1IDKj06BVIfKLwvpeaDQNanPVeCL4BreAndIsC5PMyZUCEPrGFB1ezgvsJ+Ko0ovdMMwr73/qdyTF8ZsJoxWwI20zj7doWCNSRwDjfYHibtRwDAL1/BYUNKC7sCLwCL/NrAU9RU+taCyQGtwFf7xDdYLmGZzLo1bPgcSSZFDHA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IaY2js9sqxuo853EV60UrF0MqHxpfDbmrxyb/gkQxqjci0O2JEaWExX8ttGdOYAc8XA+GVB2w7BNX89SgwwZNF3Kh+PbX/KNvUqjRQAoDE3WZCYFWcpFHw+ofn2yF3UjBauK/AuBY+DWGn3spdH3hdwWomVkntUwiu8mieohoyRNc1cx4iugKxI+Lf6ki2iEuoXbYTmoib2K1yCp3Bdtg6ACU2Eyco93xd982y2nkj4e2RJd7+JsK/7x7Erb3hJYENlDxT6x0b49YR8+Jbauy33b2zmNay82AzSdSco0VQ2xyhXL/oqBY4I0XLnN62O2zIe8+A25dvJZKLOl1Vas8Q==
  • Authentication-results: esa6.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Wei Liu <wl@xxxxxxx>, "Anthony Perard" <anthony.perard@xxxxxxxxxx>, Juergen Gross <jgross@xxxxxxxx>, "Andrew Cooper" <Andrew.Cooper3@xxxxxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, David Scott <dave@xxxxxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, "Roger Pau Monne" <roger.pau@xxxxxxxxxx>
  • Delivery-date: Thu, 27 Jan 2022 16:21:25 +0000
  • Ironport-data: A9a23:uvY+NK9vmLgjbfLiEu6WDrUD1XiTJUtcMsCJ2f8bNWPcYEJGY0x3m jAYWDyEOaqIMzGnf99/Yd+3/UMG6sLdnYMwGwJury48E34SpcT7XtnIdU2Y0wF+jyHgoOCLy +1EN7Es+ehtFie0Si9AttENlFEkvU2ybuOU5NXsZ2YhFWeIdA970Ug5w7dj3dYy6TSEK1jlV e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPhq+ upQrKKBWT0YL6yVlsI6QjN/DBpxaPguFL/veRBTsOSWxkzCNXDt3+9vHAc9OohwFuRfWD8Us 6ZCcXZUM07F17neLLGTE4GAguwoLNPqJ58es3cmyTDfAfcpaZvCX7/L9ZlT2zJYasVmQ6yOP ZtAMGsHgBLoPE1gZngaWZ8CnsSyo3ymXmAHp0irnP9ii4TU5FMoi+W8WDbPQfSRXtlclEuco mPA/kz6DwscOdjZziCKmlqum+vOkCXTSI8UUrqi+ZZCvlqXwWACDQwMYnGyq/K5l02WVspWL gof/S9GhaQ76kGDVNT2WBy85nmesXY0RN54A+A8rgaXxcL88wufQ2QJUDNFQNgnr9MtAywn0 EeTmNHkDiApt6eaIVqf/LqJqTK5OQAOMHQPIyQDSGMt4db5p5oopgnSVdslG6mw5uAZAhmpn WrM9nJnwexO04hbjM1X4GwrnRq3lKfUVzAE9j/mT2P84CVedaSCeLenvA2zAel7EK6VSVyIv X4hkseY7fwTAZzlqBFhUNnhD5nyua/bbWS0bUpHWsB4qm/zoyLLkZV4vWkmTHqFJProbtMAj KX7nQpKrKFeM3KxBUOcS9LgUp96pUQM+DmMaxw1UjasSsUpHONk1Hs3DaJ144wLuBJx+U3YE czDGftA9V5AVcxaIMOeHo/xK4MDyCEk3n/0Tpvm1Rmh2rf2TCfLFexfawPVML1hsfLsTODpH zB3bZXiJ/J3C7WWX8Uq2dRLcQBiwYYTW/gaVPC7hsbce1E7SQnN+tfawK87epwNokimvrygw 51JYWcBkACXrSSecW2iMyk/AJuyA8oXhS9lbEQEYAb5s1B+MN3HxPpOKPMKkUwPqbYLIQhcF adVIq1tw51nF1z6xtjqRcCt9dM5LEXy21vm0ujMSGFXQqOMjjfho7fMVgDu6DMPHmyws84/q KenzQTVXdwIQAEKMSocQKvHI4qZsSdPleRscVHPJ9UPKkzg/JIzc376j+MtItFKIhLGn2PI2 wGTCBYehO/Mv45qr4WZ2fHa99+kQ7lkA05XP2jH9rLqZyPUyXWunN1bW+GScDGDCG6toPe+Z f9Yxu3XOeEcmAoYqJJ1FrtmlPps59bmq7JA4B5jGXHHMwaiBr96eyHU1shTrKxdgLRevFLuC E6I/9BbP5SPOd/kTwFNdFZ0MLzb2KhNyDfI7PkzLEHr3wNN/eKKARdIIh2BqC1BN78pYokr9 vgs5ZwN4Aulhxt0btvf1nJI936BJ2ArWrk8ss1IG5fijwcmxw0QYZHYDSOqspiDZ88VbxsvK z6QwqHDm65d1gzJdH9qTSrB2u9UhJIvvhFWzQBdewTVy4Sd3vJnjgdM9TkXTxhOykQV2u1+D WFnKklpKPjc5DxvnsVCAzihFgwp6Md1IaAtJ4/lTFHkcnQ=
  • Ironport-hdrordr: A9a23:7KF9u6zec5yljtn/9ekJKrPxheskLtp133Aq2lEZdPULSKKlfp GV88jziyWZtN9IYgBdpTiBUJPwJU80hqQFnrX5XI3SEjUO3VHIEGgM1/qb/9SNIVydygcZ79 YcT0EcMqywMbEZt7eA3ODQKb9Jq7PrkNHKuQ6d9QYWcegDUdA60+4TMHf9LqQCfng+OXNPLu v72iMonUvERV0nKuCAQlUVVenKoNPG0Lj8ZwQdOhIh4A6SyRu19b/TCXGjr1cjegIK5Y1n3X nOkgT/6Knmmeq80AXg22ja6IkTsMf9y+FEGNeHhqEuW3bRY0eTFcZcso+5zXQISdKUmREXeR 730lEd1vFImjbsl6eO0ELQMkfboW4TAjTZuCGlaDPY0L3ErXQBepB8bMtiA2jkwltls9dm3K 1R2WWF85JREBPbhSz4o8PFThdwiyOP0DcfeX56tQ0ubWIyUs4bkWUkxjIeLH7AJlOM1Kk3VO 11SM3M7vdfdl2XK3jfo2l02dSpGnA+BA2PTEQOstGcl2E+pgE382IIgMgE2nsQ/pM0TJdJo+ zCL6RzjblLCssbd7h0CusNSda+TmbNXRXPOmSPJkmPLtBKB1vd75rspLkl7uCjf5IFiJM0hZ TaSVtd8XU/fkr/YPf+lKGjMiq9CVlVcQ6dv/221qIJzIEUHoCbQxFrYGpe5/ednw==
  • Ironport-sdr: Bh3KZUha1DWXa3ZPvRAZh486hM1cVNeyXaVBIKQZXMu1fPZf4YcWwjk4TS8Tn/Pg/B8LBmJxnM hp3pzP0HLobV0FaXc7PwGCcWGZ0Ucrhb/oyxzc4SxkRxtcVHlgEAGKRLleLoUBfGlQEXUiqWSx cuMWi6LRzECeCWFFKUexmCD6H4FqCzPAT11CZD0uPSf3dTi0VYFEot/R8rJFc55o/UFqCvONrx wCab1OdfS2HQoEfE34eSf3ZXSbvd6O3PMaX4Wi9pjVW/R+PBadYqujsWDqWlqPx/SbpzTSN/ia tuooGgAj1qF76HvubGCfthDj
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHYE5eVZRQ1lFhjOU+HWGCR3lWBc6x3DJuA
  • Thread-topic: [PATCH 2/2] x86/xen: Allow per-domain usage of hardware virtualized APIC

The changes for the OCaml bindings are minimal and administrative. This looks good to me.

Acked-by: Christian Lindig <christian.lindig@xxxxxxxxxx>

On 27 Jan 2022, at 16:01, Jane Malalane <jane.malalane@xxxxxxxxxx> wrote:

Introduce a new per-domain creation x86 specific flag to
select whether hardware assisted virtualization should be used for
x{2}APIC.

A per-domain option is added to xl in order to select the usage of
x{2}APIC hardware assisted vitualization, as well as a global
configuration option.

Having all APIC interaction exit to Xen for emulation is slow and can
induce much overhead. Hardware can speed up x{2}APIC by running APIC
read/write accesses without taking a VM exit.

Signed-off-by: Jane Malalane <jane.malalane@xxxxxxxxxx>
Suggested-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
---
CC: Wei Liu <wl@xxxxxxx>
CC: Anthony PERARD <anthony.perard@xxxxxxxxxx>
CC: Juergen Gross <jgross@xxxxxxxx>
CC: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
CC: George Dunlap <george.dunlap@xxxxxxxxxx>
CC: Jan Beulich <jbeulich@xxxxxxxx>
CC: Julien Grall <julien@xxxxxxx>
CC: Stefano Stabellini <sstabellini@xxxxxxxxxx>
CC: Christian Lindig <christian.lindig@xxxxxxxxxx>
CC: David Scott <dave@xxxxxxxxxx>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
CC: "Roger Pau Monné" <roger.pau@xxxxxxxxxx>
---
docs/man/xl.cfg.5.pod.in              | 10 ++++++++
docs/man/xl.conf.5.pod.in             | 12 ++++++++++
tools/golang/xenlight/helpers.gen.go  | 12 ++++++++++
tools/libs/light/libxl_arch.h         |  5 ++--
tools/libs/light/libxl_arm.c          |  5 ++--
tools/libs/light/libxl_create.c       | 21 ++++++++++-------
tools/libs/light/libxl_types.idl      |  2 ++
tools/libs/light/libxl_x86.c          | 43 +++++++++++++++++++++++++++++++++--
tools/ocaml/libs/xc/xenctrl.ml        |  2 ++
tools/ocaml/libs/xc/xenctrl.mli       |  2 ++
tools/xl/xl.c                         |  8 +++++++
tools/xl/xl.h                         |  2 ++
tools/xl/xl_parse.c                   | 14 ++++++++++++
xen/arch/x86/domain.c                 | 27 +++++++++++++++++++++-
xen/arch/x86/hvm/vmx/vmcs.c           |  4 ++++
xen/arch/x86/hvm/vmx/vmx.c            | 13 +++++++----
xen/arch/x86/include/asm/hvm/domain.h |  6 +++++
xen/arch/x86/traps.c                  |  6 +++--
xen/include/public/arch-x86/xen.h     |  2 ++
19 files changed, 174 insertions(+), 22 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index b98d161398..974fe7d2d8 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -1862,6 +1862,16 @@ firmware tables when using certain older guest Operating
Systems. These tables have been superseded by newer constructs within
the ACPI tables.

+=item B<assisted_xapic=BOOLEAN>
+B<(x86 only)> Enables or disables hardware assisted virtualization for xapic.
+This allows accessing APIC registers without a VM-exit.
+The default is settable via L<xl.conf(5)>.
+
+=item B<assisted_x2apic=BOOLEAN>
+B<(x86 only)> Enables or disables hardware assisted virtualization for x2apic.
+This allows accessing APIC registers without a VM-exit.
+The default is settable via L<xl.conf(5)>.
+
=item B<nx=BOOLEAN>

B<(x86 only)> Hides or exposes the No-eXecute capability. This allows a guest
diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in
index df20c08137..2d0a59d019 100644
--- a/docs/man/xl.conf.5.pod.in
+++ b/docs/man/xl.conf.5.pod.in
@@ -107,6 +107,18 @@ Sets the default value for the C<max_grant_version> domain config value.

Default: maximum grant version supported by the hypervisor.

+=item B<assisted_xapic=BOOLEAN>
+
+If enabled, domains will use xAPIC hardware assisted emulation by default.
+
+Default: enabled.
+
+=item B<assisted_x2apic=BOOLEAN>
+
+If enabled, domains will use x2APIC hardware assisted emulation by default.
+
+Default: enabled.
+
=item B<vif.default.script="PATH">

Configures the default hotplug script used by virtual network devices.
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index dd4e6c9f14..90e7b9b205 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -636,6 +636,12 @@ x.Passthrough = Passthrough(xc.passthrough)
if err := x.XendSuspendEvtchnCompat.fromC(&xc.xend_suspend_evtchn_compat);err != nil {
return fmt.Errorf("converting field XendSuspendEvtchnCompat: %v", err)
}
+if err := x.ArchX86.AssistedXapic.fromC(&xc.arch_x86.assisted_xapic);err != nil {
+return fmt.Errorf("converting field ArchX86.AssistedXapic: %v", err)
+}
+if err := x.ArchX86.AssistedX2Apic.fromC(&xc.arch_x86.assisted_x2apic);err != nil {
+return fmt.Errorf("converting field ArchX86.AssistedX2Apic: %v", err)
+}

 return nil}

@@ -679,6 +685,12 @@ xc.passthrough = C.libxl_passthrough(x.Passthrough)
if err := x.XendSuspendEvtchnCompat.toC(&xc.xend_suspend_evtchn_compat); err != nil {
return fmt.Errorf("converting field XendSuspendEvtchnCompat: %v", err)
}
+if err := x.ArchX86.AssistedXapic.toC(&xc.arch_x86.assisted_xapic); err != nil {
+return fmt.Errorf("converting field ArchX86.AssistedXapic: %v", err)
+}
+if err := x.ArchX86.AssistedX2Apic.toC(&xc.arch_x86.assisted_x2apic); err != nil {
+return fmt.Errorf("converting field ArchX86.AssistedX2Apic: %v", err)
+}

 return nil
 }
diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h
index 00cc50394d..2eaff45526 100644
--- a/tools/libs/light/libxl_arch.h
+++ b/tools/libs/light/libxl_arch.h
@@ -71,8 +71,9 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc,
                                               libxl_domain_create_info *c_info);

_hidden
-void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
-                                              libxl_domain_build_info *b_info);
+int libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
+                                             libxl_domain_build_info *b_info,
+                                             const libxl_physinfo *physinfo);

_hidden
int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc,
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 52f2545498..4d422bef96 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1384,8 +1384,9 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc,
    }
}

-void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
-                                              libxl_domain_build_info *b_info)
+int libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
+                                             libxl_domain_build_info *b_info,
+                                             const libxl_physinfo *physinfo)
{
    /* ACPI is disabled by default */
    libxl_defbool_setdefault(&b_info->acpi, false);
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index d7a40d7550..2bae6fef62 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -264,7 +264,6 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
    if (!b_info->event_channels)
        b_info->event_channels = 1023;

-    libxl__arch_domain_build_info_setdefault(gc, b_info);
    libxl_defbool_setdefault(&b_info->dm_restrict, false);

    if (b_info->iommu_memkb == LIBXL_MEMKB_DEFAULT)
@@ -456,15 +455,21 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
        libxl_defbool_setdefault(&b_info->nested_hvm,               false);
    }

-    if (b_info->max_grant_version == LIBXL_MAX_GRANT_DEFAULT) {
-        libxl_physinfo info;
+    libxl_physinfo info;

-        rc = libxl_get_physinfo(CTX, &info);
-        if (rc) {
-            LOG(ERROR, "failed to get hypervisor info");
-            return rc;
-        }
+    rc = libxl_get_physinfo(CTX, &info);
+    if (rc) {
+        LOG(ERROR, "failed to get hypervisor info");
+        return rc;
+    }

+    rc = libxl__arch_domain_build_info_setdefault(gc, b_info, &info);
+    if (rc) {
+        LOG(ERROR, "unable to set domain arch build info defaults");
+        return rc;
+    }
+
+    if (b_info->max_grant_version == LIBXL_MAX_GRANT_DEFAULT) {
        if (info.cap_gnttab_v2)
            b_info->max_grant_version = 2;
        else if (info.cap_gnttab_v1)
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 42ac6c357b..db5eb0a0b3 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -648,6 +648,8 @@ libxl_domain_build_info = Struct("domain_build_info",[
                               ("vuart", libxl_vuart_type),
                              ])),
    ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool),
+                               ("assisted_xapic", libxl_defbool),
+                               ("assisted_x2apic", libxl_defbool),
                              ])),
    # Alternate p2m is not bound to any architecture or guest type, as it is
    # supported by x86 HVM and ARM support is planned.
diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
index 33da51fe89..b257fca756 100644
--- a/tools/libs/light/libxl_x86.c
+++ b/tools/libs/light/libxl_x86.c
@@ -23,6 +23,12 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
    if (libxl_defbool_val(d_config->b_info.arch_x86.msr_relaxed))
        config->arch.misc_flags |= XEN_X86_MSR_RELAXED;

+    if(libxl_defbool_val(d_config->b_info.arch_x86.assisted_xapic))
+        config->arch.misc_flags |= XEN_X86_ASSISTED_XAPIC;
+
+    if(libxl_defbool_val(d_config->b_info.arch_x86.assisted_x2apic))
+        config->arch.misc_flags |= XEN_X86_ASSISTED_X2APIC;
+
    return 0;
}

@@ -819,11 +825,44 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc,
{
}

-void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
-                                              libxl_domain_build_info *b_info)
+int libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
+                                             libxl_domain_build_info *b_info,
+                                             const libxl_physinfo *physinfo)
{
+    int rc;
+    bool assisted_xapic;
+    bool assisted_x2apic;
+
    libxl_defbool_setdefault(&b_info->acpi, true);
    libxl_defbool_setdefault(&b_info->arch_x86.msr_relaxed, false);
+
+    libxl_defbool_setdefault(&b_info->arch_x86.assisted_xapic, false);
+    libxl_defbool_setdefault(&b_info->arch_x86.assisted_x2apic, false);
+
+    assisted_xapic = libxl_defbool_val(b_info->arch_x86.assisted_xapic);
+    assisted_x2apic = libxl_defbool_val(b_info->arch_x86.assisted_x2apic);
+
+    if ((assisted_xapic || assisted_x2apic) &&
+        b_info->type == LIBXL_DOMAIN_TYPE_PV)
+    {
+        LOG(ERROR, "Interrupt Controller Virtualization not supported for PV");
+        rc = ERROR_INVAL;
+        goto out;
+    }
+
+    if ((assisted_xapic && !physinfo->cap_assisted_xapic) ||
+         (assisted_x2apic && !physinfo->cap_assisted_x2apic))
+    {
+        LOG(ERROR, "x%sAPIC hardware supported emulation not available",
+            assisted_xapic && !physinfo->cap_assisted_xapic ? "" : "2");
+        rc =  ERROR_INVAL;
+        goto out;
+    }
+
+    rc = 0;
+out:
+    return rc;
+
}

int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc,
diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index 7ce832d605..cce30d8731 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -50,6 +50,8 @@ type x86_arch_emulation_flags =

type x86_arch_misc_flags =
| X86_MSR_RELAXED
+ | X86_ASSISTED_XAPIC
+ | X86_ASSISTED_X2APIC

type xen_x86_arch_domainconfig =
{
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index a2b15130ee..67a22ec15c 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -44,6 +44,8 @@ type x86_arch_emulation_flags =

type x86_arch_misc_flags =
  | X86_MSR_RELAXED
+  | X86_ASSISTED_XAPIC
+  | X86_ASSISTED_X2APIC

type xen_x86_arch_domainconfig = {
  emulation_flags: x86_arch_emulation_flags list;
diff --git a/tools/xl/xl.c b/tools/xl/xl.c
index 2d1ec18ea3..b97e491c9c 100644
--- a/tools/xl/xl.c
+++ b/tools/xl/xl.c
@@ -57,6 +57,8 @@ int max_grant_frames = -1;
int max_maptrack_frames = -1;
int max_grant_version = LIBXL_MAX_GRANT_DEFAULT;
libxl_domid domid_policy = INVALID_DOMID;
+int assisted_xapic = 0;
+int assisted_x2apic = 0;

xentoollog_level minmsglevel = minmsglevel_default;

@@ -201,6 +203,12 @@ static void parse_global_config(const char *configfile,
    if (!xlu_cfg_get_long (config, "claim_mode", &l, 0))
        claim_mode = l;

+    if (!xlu_cfg_get_long (config, "assisted_xapic", &l, 0))
+        assisted_xapic = l;
+
+    if (!xlu_cfg_get_long (config, "assisted_x2apic", &l, 0))
+        assisted_x2apic = l;
+
    xlu_cfg_replace_string (config, "remus.default.netbufscript",
        &default_remus_netbufscript, 0);
    xlu_cfg_replace_string (config, "colo.default.proxyscript",
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index c5c4bedbdd..528deb3feb 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -286,6 +286,8 @@ extern libxl_bitmap global_vm_affinity_mask;
extern libxl_bitmap global_hvm_affinity_mask;
extern libxl_bitmap global_pv_affinity_mask;
extern libxl_domid domid_policy;
+extern int assisted_xapic;
+extern int assisted_x2apic;

enum output_format {
    OUTPUT_FORMAT_JSON,
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 117fcdcb2b..16ff9e76bc 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1681,6 +1681,20 @@ void parse_config_data(const char *config_source,
        xlu_cfg_get_defbool(config, "vpt_align", &b_info->u.hvm.vpt_align, 0);
        xlu_cfg_get_defbool(config, "apic", &b_info->apic, 0);

+        e = xlu_cfg_get_defbool(config, "assisted_xapic",
+                                &b_info->arch_x86.assisted_xapic, 0);
+        if (e == ESRCH) /* not specified */
+            libxl_defbool_set(&b_info->arch_x86.assisted_xapic, assisted_xapic);
+        else if (e)
+            exit(1);
+
+        e = xlu_cfg_get_defbool(config, "assisted_x2apic",
+                                &b_info->arch_x86.assisted_x2apic, 0);
+        if (e == ESRCH) /* not specified */
+            libxl_defbool_set(&b_info->arch_x86.assisted_x2apic, assisted_x2apic);
+        else if (e)
+            exit(1);
+
        switch (xlu_cfg_get_list(config, "viridian",
                                 &viridian, &num_viridian, 1))
        {
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ef1812dc14..d08f51e28b 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -619,6 +619,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
    bool hvm = config->flags & XEN_DOMCTL_CDF_hvm;
    bool hap = config->flags & XEN_DOMCTL_CDF_hap;
    bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt;
+    bool assisted_xapic = config->arch.misc_flags & XEN_X86_ASSISTED_XAPIC;
+    bool assisted_x2apic = config->arch.misc_flags & XEN_X86_ASSISTED_X2APIC;
    unsigned int max_vcpus;

    if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
@@ -685,13 +687,30 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
        }
    }

-    if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED )
+    if ( config->arch.misc_flags & ~(XEN_X86_MSR_RELAXED |
+                                     XEN_X86_ASSISTED_XAPIC |
+                                     XEN_X86_ASSISTED_X2APIC) )
    {
        dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n",
                config->arch.misc_flags);
        return -EINVAL;
    }

+    if ( (assisted_xapic || assisted_x2apic) && !hvm )
+    {
+        dprintk(XENLOG_INFO,
+                "Interrupt Controller Virtualization not supported for PV\n");
+        return -EINVAL;
+    }
+
+    if ( (assisted_xapic && !assisted_xapic_available) ||
+         (assisted_x2apic && !assisted_x2apic_available) )
+    {
+        dprintk(XENLOG_INFO, "x%sAPIC requested but not available\n",
+                assisted_xapic && !assisted_xapic_available ? "" : "2");
+        return -EINVAL;
+    }
+
    return 0;
}

@@ -863,6 +882,12 @@ int arch_domain_create(struct domain *d,

    d->arch.msr_relaxed = config->arch.misc_flags & XEN_X86_MSR_RELAXED;

+    d->arch.hvm.assisted_xapic =
+        config->arch.misc_flags & XEN_X86_ASSISTED_XAPIC;
+
+    d->arch.hvm.assisted_x2apic =
+        config->arch.misc_flags & XEN_X86_ASSISTED_X2APIC;
+
    return 0;

 fail:
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 7691db0fdd..cb2d9759c9 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -1154,6 +1154,10 @@ static int construct_vmcs(struct vcpu *v)
        __vmwrite(PLE_WINDOW, ple_window);
    }

+    if ( !v->domain->arch.hvm.assisted_xapic )
+        v->arch.hvm.vmx.secondary_exec_control &=
+            ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES;
+
    if ( cpu_has_vmx_secondary_exec_control )
        __vmwrite(SECONDARY_VM_EXEC_CONTROL,
                  v->arch.hvm.vmx.secondary_exec_control);
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index c44cf8f5d4..6501fb1a08 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3342,16 +3342,19 @@ static void vmx_install_vlapic_mapping(struct vcpu *v)

void vmx_vlapic_msr_changed(struct vcpu *v)
{
-    int virtualize_x2apic_mode;
+    int virtualize_xapic_mode, virtualize_x2apic_mode;
    struct vlapic *vlapic = vcpu_vlapic(v);
    unsigned int msr;

+    virtualize_xapic_mode = ( cpu_has_vmx_virtualize_apic_accesses &&
+                              v->domain->arch.hvm.assisted_xapic );
+
    virtualize_x2apic_mode = ( (cpu_has_vmx_apic_reg_virt ||
                                cpu_has_vmx_virtual_intr_delivery) &&
-                               cpu_has_vmx_virtualize_x2apic_mode );
+                               cpu_has_vmx_virtualize_x2apic_mode &&
+                               v->domain->arch.hvm.assisted_x2apic );

-    if ( !cpu_has_vmx_virtualize_apic_accesses &&
-         !virtualize_x2apic_mode )
+    if ( !virtualize_xapic_mode && !virtualize_x2apic_mode )
        return;

    vmx_vmcs_enter(v);
@@ -3382,7 +3385,7 @@ void vmx_vlapic_msr_changed(struct vcpu *v)
                vmx_clear_msr_intercept(v, MSR_X2APIC_SELF, VMX_MSR_W);
            }
        }
-        else
+        else if ( virtualize_xapic_mode )
            v->arch.hvm.vmx.secondary_exec_control |=
                SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES;
    }
diff --git a/xen/arch/x86/include/asm/hvm/domain.h b/xen/arch/x86/include/asm/hvm/domain.h
index 698455444e..0e888cbeba 100644
--- a/xen/arch/x86/include/asm/hvm/domain.h
+++ b/xen/arch/x86/include/asm/hvm/domain.h
@@ -117,6 +117,12 @@ struct hvm_domain {

    bool                   is_s3_suspended;

+    /* xAPIC hardware assisted emulation. */
+    bool assisted_xapic;
+
+    /* x2APIC hardware assisted emulation. */
+    bool assisted_x2apic;
+
    /* hypervisor intercepted msix table */
    struct list_head       msixtbl_list;

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 485bd66971..c215605814 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -1115,7 +1115,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf,
        if ( !is_hvm_domain(d) || subleaf != 0 )
            break;

-        if ( cpu_has_vmx_apic_reg_virt )
+        if ( cpu_has_vmx_apic_reg_virt &&
+             v->domain->arch.hvm.assisted_xapic )
            res->a |= XEN_HVM_CPUID_APIC_ACCESS_VIRT;

        /*
@@ -1126,7 +1127,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf,
         */
        if ( cpu_has_vmx_virtualize_x2apic_mode &&
             cpu_has_vmx_apic_reg_virt &&
-             cpu_has_vmx_virtual_intr_delivery )
+             cpu_has_vmx_virtual_intr_delivery &&
+             v->domain->arch.hvm.assisted_x2apic )
            res->a |= XEN_HVM_CPUID_X2APIC_VIRT;

        /*
diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
index f7b17e0f98..3c63c027a8 100644
--- a/xen/include/public/arch-x86/xen.h
+++ b/xen/include/public/arch-x86/xen.h
@@ -317,6 +317,8 @@ struct xen_arch_domainconfig {
 * doesn't allow the guest to read or write to the underlying MSR.
 */
#define XEN_X86_MSR_RELAXED (1u << 0)
+#define XEN_X86_ASSISTED_XAPIC (1u << 1)
+#define XEN_X86_ASSISTED_X2APIC (1u << 2)
    uint32_t misc_flags;
};

--
2.11.0



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.