[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 1/3] IOMMU: record the flushes skipped under iommu_dont_flush_iotlb


  • To: Luca Di Maio <luca@xxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx
  • From: Teddy Astie <teddy.astie@xxxxxxxxxx>
  • Date: Sat, 31 Oct 2026 11:55:17 +0100
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=selector1 header.d=vates.tech header.i="@vates.tech" header.h="From:Subject:Date:Message-ID:To:Cc:MIME-Version:Content-Type:In-Reply-To:References:Feedback-ID"
  • Autocrypt: addr=teddy.astie@xxxxxxxxxx; keydata= xsDNBGn5sK8BDACuzSrrTjpVf4ay06OYB6yY0J1PqKffihoNMtrQRZjAHxoAPC7LTBVHV/XO Zw5HJc+9R71z1JV+iYg6z3jPziGKzX8Fj3ZXlzJPmpf1PuETH3KdbvtJT4ny+OGntnJntUoR KRPhTirr6yNeBk/637O3CQXjtqFUPZnko8OI/o1yawIBhJJAWicutjkkUgd28Bh6HV9EIumH tCBgn5/1A/fpm9624MMgYLsA8qjC4XsoovQvFCaO8HEhvfzrrTZHjn/nPeB9SigxIxXW8YaT VqMdqul07o72m3eA2mf+LMu9a04FX/d4wbxBLtELm+1jIrbtyaFZEMOLv/haSiS/Lj3btJH/ EoucejoZ5SH49ksmVAmKOLktOaTQ8b2gEvP7iaKiIiszCCtOSRohr+2GvDsDeLvVZnlR3I+S PhHar7TPKjFz0G3DPNolyjXywNqOAMpomSPi8lSwjAFsxOtQbcck/qRGRSNk4DAmH70pA+89 MXfQXZ3qt1Q01B1+sU0I8xsAEQEAAc0kVGVkZHkgQXN0aWUgPHRlZGR5LmFzdGllQHZhdGVz LnRlY2g+wsENBBMBCAA3FiEEGAIew9LzHY3pdrqtZg+p0QLLz9AFAmn5sK8FCQWjmoACGwME CwkIBwUVCAkKCwUWAgMBAAAKCRBmD6nRAsvP0ID6DACGOktArFbLKHNzuyOVCskwfUZPla6Z pd3GZ8r61SrAKePIr2BnpgPkd0hV3bSRkRLIrgjzR2NRCzfp0x0HfuhcYfAYPR46XHTvjaJE v99sT/vGUG1BZguYDOScSEpgSNaNlYum3RKZbMuROxdK8G+YHccJY8PvWSq2K2yiae2KGiAv 1yjnZxug9/PtDfX8vQFUSg2w1ukRDf50wvDohN1zUQfFtofOP2xCRsDZiHAlQ0pF+aUjXQhP eP3IdpfWc8cyRLXF06Rk46YMYCytweGtGdHcqAfrVthl84129ZPN422k/voW0sm14gjYlGcT UwgnYlFRk2FLq0QeKEDcS0aj3o3EVAQCrayoGzi1pnlIKE3PRGUcUzjGVvzQ/po24gOjwba9 Egr/Wmu3MQlx/7A8zT5QBzF/n+RYdLNQ0Eu6YnUwf0Z1uieqNaon+olyIRFiLb/hCZHO6ekN f5vrm2clHUbQAYaPQebknujoKBo6ZLHg0WM1gZS01Gz+aUpKsUfOwM0EafmwsAEMAKiQiZa3 yQMmc/h3sDbfVHPSiBA4IMI/NAB7IotzPHq1GzCpsoVILAhF/INbWjxJ3DbVf+en3/FvdVZg 2S38xtnth0njNdlVKpyxm054phKjbdoFDwaknWolS4hrddTmetSG5/52AjtmPFtlXAk0NmLv fJnW3seXVQbgM7sW/MNXPP5UKDpkGnLhnvej+GU0s3109sJeXT5ImVdphFs9cvyZyBT9t1Pb Rowv58EgV0zE4hbAeVkULAbxFV5b/ExTjjGVHoX7CVhWxvCiTqCUoXZRkUE9C3FnkzEFRkKb Yu6NCfiHfEyB3Xyg9hfdrRgjMRq907zCof+nDtWxGz1MSEuvTj1g9GZ049Bennqzjc/Q+0ov XoK4jm+Py0FiUGUaA6yhexficjH+kCR/xDbVnWrMhSLB4AuTBT9HjfZI6gk3uYLhoT8Pig4/ eVtR2Q1wZIJsFToR6ofGuyECwFcs+PUXN7fmGRSiPXgjAr/zIUBdW0VWCE3OGPNqtRk2E5s6 IQARAQABwsD8BBgBCAAmFiEEGAIew9LzHY3pdrqtZg+p0QLLz9AFAmn5sLAFCQWjmoACGwwA CgkQZg+p0QLLz9DncQwAg76IehTemLIfrB8T9WIBZrI4kUV7G7a4rjiVoUiHYN5QwhnbZnsa JDlt+Ezoqy/510eo2bCSzvW5xXYPgyjcuOPwgQo1Qp764QxyX6rld2f2RcWkDuBHun55ZWXj by8o21ginPRwruBVYY5rVf3DV1iBu4NurUeHtyFk/dS0XTOQi2wVUb17sW/+ybCEokdVacZG zOqP/OmwHrF8ylXlXnhQq6e3r+J+T8fuoGJelm/CJiMwyP6cEWE8sxVqX/iqwjwUYkuOCpE+ lOWSvdNHgoEkWR0RXBPQjnGmLKbfTl/QDXLk6NP2/r9uxm2HL6Ei3QJKSEdrp+XZaVnk/Off O485NOTKwGOxyWb006cTMh53xPkAJFQu4Tvdj+odsHz88jqw5wfPG0BYWx0I/FspYj7N9kZR 8ULR9nX0LvpzJ/kB4NgHIUt8YtIL6ZSfM2dbF7fKzvx1UqFfvozJZwFzfEieJLXa4nlGgR6D x9fhaZEsniw8/bYgC3igkk5YJiOa
  • Cc: Jan Beulich <jbeulich@xxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Roger Pau Monné <roger@xxxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Bertrand Marquis <bertrand.marquis@xxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>
  • Delivery-date: Sat, 31 Oct 2026 10:55:44 +0000
  • Feedback-id: default:8631fc262581453bbf619ec5b2062170:Sweego
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

Le 30/10/2026 à 17:53, Luca Di Maio a écrit :
While iommu_dont_flush_iotlb is set, iommu_legacy_unmap() and the
shared-EPT path of ept_set_entry() skip their IOTLB flush and drop the
flags it would have used. Accumulate them in the per-CPU
iommu_skipped_flush_flags, so that a caller deferring the flush can
issue it with the same flags. The next patch does so for grant unmaps.

Use a uint8_t, as a fixed width is meant: a byte holds every flag and
fits in the padding after iommu_dont_flush_iotlb, so the per-CPU data
that follows doesn't move.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Luca Di Maio <luca@xxxxxxxxx>

---
  xen/arch/x86/mm/p2m-ept.c       |  4 ++++
  xen/drivers/passthrough/iommu.c | 12 +++++++++++-
  xen/include/xen/iommu.h         |  1 +
  3 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 204bdb054a..1a87b25d94 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -981,6 +981,10 @@ out:
                                     (iommu_flags ? IOMMU_FLUSHF_added : 0) |
                                     (vtd_pte_present ? IOMMU_FLUSHF_modified
                                                      : 0));
+        else if ( iommu_use_hap_pt(d) )
+            this_cpu(iommu_skipped_flush_flags) |=
+                (iommu_flags ? IOMMU_FLUSHF_added : 0) |
+                (vtd_pte_present ? IOMMU_FLUSHF_modified : 0);
          else if ( need_iommu_pt_sync(d) )
              rc = iommu_flags ?
                  iommu_legacy_map(d, _dfn(gfn), mfn, 1ul << order, 
iommu_flags) :

We now have

(iommu_flags ? IOMMU_FLUSHF_added : 0) |
(vtd_pte_present ? IOMMU_FLUSHF_modified : 0)

duplicated in 2 different (adjacent) places, I think we should collapse it in a variable. i.e something like

unsigned long flush_flags =
    (iommu_flags ? IOMMU_FLUSHF_added : 0) |
    (vtd_pte_present ? IOMMU_FLUSHF_modified : 0);

But more generally, given that iommu_dont_flush_iotlb is only set/cleared in paths that check it (directly or indirectly), what about moving this logic into iommu_iotlb_flush() to be suppressed by iommu_dont_flush_iotlb (and if suppressed, |= the iommu_skipped_flush_flags instead) ?

That would get rid of a lot of duplicated logic.

diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 4b22ecfe65..70ee45afb9 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -64,6 +64,11 @@ bool __read_mostly iommu_hap_pt_share = true;
  bool __read_mostly iommu_debug;
DEFINE_PER_CPU(bool, iommu_dont_flush_iotlb);
+/*
+ * A byte holds every IOMMU_FLUSHF_*, and fits in the alignment padding the
+ * bool above leaves, instead of shifting the per-CPU data that follows.
+ */
+DEFINE_PER_CPU(uint8_t, iommu_skipped_flush_flags);

flush_flags in iotlb_flush is unsigned int in iommu_iotlb_flush(), so I would use it here as well instead of uint8_t.

  static int __init cf_check parse_iommu_param(const char *s)
  {
@@ -458,7 +463,12 @@ int iommu_legacy_unmap(struct domain *d, dfn_t dfn, 
unsigned long page_count)
      unsigned int flush_flags = 0;
      int rc = iommu_unmap(d, dfn, page_count, 0, &flush_flags);
- if ( !this_cpu(iommu_dont_flush_iotlb) && !rc )
+    if ( this_cpu(iommu_dont_flush_iotlb) )
+    {
+        BUILD_BUG_ON(IOMMU_FLUSHF_all > UINT8_MAX);
+        this_cpu(iommu_skipped_flush_flags) |= flush_flags;
+    }
+    else if ( !rc )
          rc = iommu_iotlb_flush(d, dfn, page_count, flush_flags);
return rc;
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 37c4a1dc82..028f41bc15 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -501,6 +501,7 @@ void iommu_dev_iotlb_flush_timeout(struct domain *d, struct 
pci_dev *pdev);
   * the caller.
   */
  DECLARE_PER_CPU(bool, iommu_dont_flush_iotlb);
+DECLARE_PER_CPU(uint8_t, iommu_skipped_flush_flags);
bool arch_iommu_use_permitted(const struct domain *d);

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.