[PATCH v5 00/12] Add initial Xen Suspend-to-RAM support on ARM64

[Mykola Kvach <xakep.amatop@xxxxxxxxx>]

From: Mykola Kvach <mykola_kvach@xxxxxxxx>

This is part 2 of version 5 of the ARM Xen system suspend/resume patch
series, based on earlier work by Mirela Simonovic and Mykyta Poturai.
This version is ported to Xen master (4.21-unstable) and includes
extensive improvements based on reviewer feedback. The patch series
restructures code to improve robustness, maintainability, and implements
system Suspend-to-RAM support on ARM64 hardware domains.

At a high-level, this patch series provides:
 - Support for Host system suspend/resume via PSCI SYSTEM_SUSPEND (ARM64)
 - Suspend/resume infrastructure for CPU context, timers, GICv2/GICv3 and 
IPMMU-VMSA
 - Proper error propagation and recovery throughout the suspend/resume flow

Key updates in this series:
 - Introduced architecture-specific suspend/resume infrastructure (new 
`suspend.c`, `suspend.h`, low-level context save/restore in `head.S`)
 - Integrated GICv2/GICv3 suspend and resume, including memory-backed context 
save/restore with error handling
 - Added time and IRQ suspend/resume hooks, ensuring correct timer/interrupt 
state across suspend cycles
 - Implemented proper PSCI SYSTEM_SUSPEND invocation and version checks
 - Improved state management and recovery in error cases during suspend/resume
 - Added support for IPMMU-VMSA context save/restore

---
TODOs:
 - Test system suspend with llc_coloring_enabled set and verify functionality
 - Implement SMMUv3 suspend/resume handlers
 - Enable "xl suspend" support on ARM
 - Properly disable Xen timer watchdog from relevant services (only init.d left)
 - Add suspend/resume CI test for ARM (QEMU if feasible)
 - Investigate feasibility and need for implementing system suspend on ARM32
---
Changes introduced in V5:
 - Add support for IPMMU-VMSA context save/restore
 - Add support for GICv3 context save/restore
 - Select HAS_SYSTEM_SUSPEND in ARM_64 instead of ARM
 - Check llc_coloring_enabled instead of LLC_COLORING during the selection
   of HAS_SYSTEM_SUSPEND config
 - Call host_system_suspend from guest PSCI system suspend instead of
   arch_domain_shutdown, reducing the complexity of the new code

Changes introduced in V4:
  - Remove the prior tasklet-based workaround in favor of a more
    straightforward and safer solution.
  - Rework the approach by adding explicit system state checks around
    request_irq and release_irq calls; skip these calls during suspend
    and resume states to avoid unsafe memory operations when IRQs are
    disabled.
  - Prevent reinitialization of local IRQ descriptors on system resume.
  - Restore the state of local IRQs during system resume for secondary CPUs.
  - Drop code for saving and restoring VCPU context (see part 1 of the patch
    series for details).
  - Remove IOMMU suspend and resume calls until these features are implemented.
  - Move system suspend logic to arch_domain_shutdown, invoked from
    domain_shutdown.
  - Add console_end_sync to the resume path after system suspend.
  - Drop unnecessary DAIF masking; interrupts are already masked on resume.
  - Remove leftover TLB flush instructions; flushing is handled in enable_mmu.
  - Avoid setting x19 in hyp_resume as it is not required.
  - Replace prepare_secondary_mm with set_init_ttbr, and call it from 
system_suspend.
  - Produce a build-time error for ARM32 when CONFIG_SYSTEM_SUSPEND is enabled.
  - Use register_t instead of uint64_t in the cpu_context structure.
  - Apply minor fixes such as renaming functions, updating comments, and
    modifying commit messages to accurately reflect the changes introduced
    by this patch series.

For earlier changelogs, please refer to the previous cover letters.

Previous versions:
  V1: https://marc.info/?l=xen-devel&m=154202231501850&w=2
  V2: https://marc.info/?l=xen-devel&m=166514782207736&w=2
  V3: https://lists.xen.org/archives/html/xen-devel/2025-03/msg00168.html

Mirela Simonovic (6):
  xen/arm: Add suspend and resume timer helpers
  xen/arm: gic-v2: Implement GIC suspend/resume functions
  xen/arm: Add support for system suspend triggered by hardware domain
  xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface)
  xen/arm: Resume memory management on Xen resume
  xen/arm: Save/restore context on suspend/resume

Mykola Kvach (4):
  xen/arm: gic-v3: Implement GICv3 suspend/resume functions
  xen/arm: Prevent crash during disable_nonboot_cpus on suspend
  xen/arm: irq: avoid local IRQ descriptors reinit on system resume
  xen/arm: irq: Restore state of local IRQs during system resume

Oleksandr Tyshchenko (2):
  iommu/ipmmu-vmsa: Implement suspend/resume callbacks
  xen/arm: Suspend/resume IOMMU on Xen suspend/resume

 xen/arch/arm/Kconfig                     |   1 +
 xen/arch/arm/Makefile                    |   1 +
 xen/arch/arm/arm64/head.S                | 113 ++++++++++
 xen/arch/arm/gic-v2.c                    | 154 +++++++++++++
 xen/arch/arm/gic-v3.c                    | 233 ++++++++++++++++++++
 xen/arch/arm/gic.c                       |  35 +++
 xen/arch/arm/include/asm/gic.h           |  12 +
 xen/arch/arm/include/asm/mm.h            |   2 +
 xen/arch/arm/include/asm/psci.h          |   1 +
 xen/arch/arm/include/asm/suspend.h       |  44 ++++
 xen/arch/arm/include/asm/time.h          |   5 +
 xen/arch/arm/irq.c                       |  48 ++++
 xen/arch/arm/mmu/smpboot.c               |   2 +-
 xen/arch/arm/psci.c                      |  23 +-
 xen/arch/arm/suspend.c                   | 196 +++++++++++++++++
 xen/arch/arm/tee/ffa_notif.c             |   2 +-
 xen/arch/arm/time.c                      |  56 ++++-
 xen/arch/arm/vpsci.c                     |  12 +-
 xen/common/domain.c                      |   4 +
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 269 +++++++++++++++++++++++
 20 files changed, 1197 insertions(+), 16 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/suspend.h
 create mode 100644 xen/arch/arm/suspend.c

-- 
2.48.1