|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [PATCH 00/25 v6] SBSA UART emulation support in Xen
Hi Julien, The issue seems to be that domain_vpl011_deinit() is not getting called when we destroy the domain. On further debugging, i found that domain_relinquish_resources was failing in domain_kill() function. I believe that issue could be that arch_domain_destroy() (from where domain_vpl011_deinit() is called) is called later and because domain_kill() fails, it is not getting called. Once I moved the domain_vpl011_deinit() inside domain_kill(), the domain is getting cleaned up properly. Is it ok to call domain_vpl011_deinit() inside domain_kill? Regards, Bhupinder On 19 July 2017 at 20:57, Julien Grall <julien.grall@xxxxxxx> wrote: > Hi Bhupinder, > > I've tried this series today on an ARM64 platform. When I enable pl011 for > the guest, > I am not able to fully destroy the guest. It stay in zombie mode: > > 42sh> xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 3072 2 r----- > 62.1 > (null) 6 0 2 --p--d > 1.5 > > It does not happen when I don't have pl011 enabled in the guest config. > > The step to reproduce it is: > > 42sh> xl create guest.cfg > Parsing config from guest.cfg > 42sh> xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 3072 2 r----- > 12.9 > guest 1 3265 1 r----- > 0.1 > 42sh> xl destroy guest > 42sh> xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 3072 2 r----- > 14.9 > (null) 1 0 1 --p--d > 0.1 > > And my guest.cfg is: > > 42sh> cat guest.cfg > kernel="/home/julien/works/guest/Image" > name="guest" > memory=3265 > vcpus=2 > vuart="sbsa_uart" > > I haven't dug into the problem but I would look at how you unmap the ring > from Xen > and xenconsole. Likely we still have a reference on it. > > Let me know if you need any help. > > Cheers, > > On 17/07/17 14:06, Bhupinder Thakur wrote: >> SBSA UART emulation for guests in Xen >> ====================================== >> Linaro has published VM System specification for ARM Processors, which >> provides a set of guidelines for both guest OS and hypervisor >> implementations, >> such that building OS images according to these guidelines guarantees >> that those images can also run on hypervisors compliant with this >> specification. >> >> One of the spec requirements is that the hypervisor must provide an >> emulated SBSA UART as a serial console which meets the minimum requirements >> in >> SBSA UART as defined in appendix B of the following >> ARM Server Base Architecture Document: >> >> https://static.docs.arm.com/den0029/a/Server_Base_System_Architecture_v3_1_ARM_DEN_0029A.pdf. >> >> This feature allows the Xen guests to use SBSA compliant pl011 UART as >> as a console. >> >> Note that SBSA pl011 UART is a subset of full featured ARM pl011 UART and >> supports only a subset of registers as mentioned below. It does not support >> rx/tx DMA. >> >> Currently, Xen supports paravirtualized (aka PV console) and an emulated >> serial >> consoles. This feature will expose an emulated SBSA pl011 UART console to the >> guest, which a user can access using xenconsole. >> >> The device tree passed to the guest VM will contain the pl011 MMIO address >> range and an irq for receiving rx/tx pl011 interrupts. The device tree format >> is specified in Documentation/devicetree/bindings/serial/arm_sbsa_uart.txt. >> >> The Xen hypervisor will expose two types of interfaces to the backend and >> domU. >> >> The interface exposed to domU will be an emulated pl011 UART by emulating the >> access to the following pl011 registers by the guest. >> >> - Data register (DR) - RW >> - Raw interrupt status register (RIS) - RO >> - Masked interrupt status register (MIS)- RO >> - Interrupt Mask (IMSC) - RW >> - Interrupt Clear (ICR) - WO >> >> It will also inject the pl011 interrupts to the guest in the following >> conditions: >> >> - incoming data in the rx buffer for the guest >> - there is space in the tx buffer for the guest to write more data >> >> The interface exposed to the backend will be the same PV console interface, >> which minimizes the changes required in xenconsole to support a new pl011 >> console. >> >> This interface has rx and tx ring buffers and an event channel for >> sending/receiving events from the backend. >> >> So essentially Xen handles the data on behalf of domU and the backend. Any >> data >> written by domU is captured by Xen and written to the TX (OUT) ring buffer >> and a pl011 event is raised to the backend to read the TX ring buffer. >> >> Similarly on reciving a pl011 event, Xen injects an interrupt to guest to >> indicate there is data available in the RX (IN) ring buffer. >> >> The pl011 UART state is completely captured in the set of registers >> mentioned above and this state is updated everytime there is an event from >> the backend or there is register read/write access from domU. >> >> For example, if domU has masked the rx interrupt in the IMSC register, then >> Xen >> will not inject an interrupt to guest and will just update the RIS register. >> Once the interrupt is unmasked by guest, the interrupt will be delivered to >> the >> guest. >> >> Changes summary: >> >> Xen Hypervisor >> =============== >> >> 1. Add emulation code to emulate read/write access to pl011 registers and >> pl011 >> interrupts: >> - It emulates DR read/write by reading and writing from/to the IN and >> OUT ring buffers and raising an event to dom0 when there is data in >> the OUT ring buffer and injecting an interrupt to the guest when there >> is data in the IN ring buffer. >> - Other registers are related to interrupt management and essentially >> control when interrupts are delivered to the guest. >> >> 2. Add a new domctl API to initialize vpl011 emulation in Xen. >> >> 3. Enable vpl011 emulation for a domain based on a libxl option passed during >> domain creation. >> >> Toolstack >> ========== >> >> 1. Add a new option "vuart" in the domU configuration file to enable/disable >> vuart. >> >> 2. Create a SBSA UART DT node in the guest device tree. It uses a fixed >> vpl011 SPI IRQ number and MMIO address. >> >> 3. Call vpl011 init DOMCTL API to enable vpl011 emulation. >> >> 5. Add a new vuart xenstore node, which contains: >> - ring-ref >> - event channel >> - buffer limit >> - type >> >> Xenconsoled >> ============ >> >> 1. Split the domain structure to support multiple consoles. >> >> 2. Modify different APIs such as buffer_append() etc. to operate on the >> console structure. >> >> 3. Add support for handling multiple consoles. >> >> 4. Add support for vuart console: >> >> The vpl011 changes available at the following repo: >> >> url: https://git@xxxxxxxxxxxxxx:/people/bhupinder.thakur/xen.git >> branch: vpl011_v6 >> >> Kindly wait for one day to checkout the code from the above URL. >> >> There are some TBD items which need to be looked at in the future: >> >> 1. Currently UEFI firmware logs the output to hvc console only. How can >> UEFI firmware be made aware of pl011 console and how it can use it >> as a console instead of hvc. >> >> There was a discussion on this and it was decided that SBSA UART should >> be used as a debug port by the UEFI firmware so that all debug output >> is redirected to this port. >> >> 2. Linux seems to have hvc console as the default console i.e. if no >> console is specified then it uses hvc as the console. How can an >> option be provided in Linux to select either hvc or pl011 as the >> default console. >> >> It was suggeted to use the SPCR in ACPI and the stdout-path option in the >> device tree to specify the default console. However, currently hvc console >> is not describable in the ACPI/device tree. This support will have to be >> added to allow the user to specify the default console. >> >> 3. ACPI support for pl011 device. >> >> CC: Ian Jackson <ian.jackson@xxxxxxxxxxxxx> >> CC: Wei Liu <wei.liu2@xxxxxxxxxx> >> CC: Stefano Stabellini <sstabellini@xxxxxxxxxx> >> CC: Julien Grall <julien.grall@xxxxxxx> >> CC: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> >> >> >> Bhupinder Thakur (25): >> xen/arm: vpl011: Define common ring buffer helper functions in >> console.h >> xen/arm: vpl011: Add SBSA UART emulation in Xen >> xen/arm: vpl011: Allocate a new GFN in the toolstack for vuart >> xen/arm: vpl011: Add support for vuart in libxl >> xen/arm: vpl011: Rearrange xen header includes in alphabetical order >> in domctl.c >> xen/arm: vpl011: Add a new domctl API to initialize vpl011 >> xen/arm: vpl011: Add a new vuart node in the xenstore >> xen/arm: vpl011: Modify xenconsole to define and use a new console >> structure >> xen/arm: vpl011: Rename the console structure field conspath to xspath >> xen/arm: vpl011: Modify xenconsole functions to take console structure >> as input >> xen/arm: vpl011: Add a new console_init function in xenconsole >> xen/arm: vpl011: Add a new buffer_available function in xenconsole >> xen/arm: vpl011: Add a new add_console_evtchn_fd function in >> xenconsole >> xen/arm: vpl011: Add a new add_console_tty_fd function in xenconsole >> xen/arm: vpl011: Add a new console_evtchn_unmask function in >> xenconsole >> xen/arm: vpl011: Add a new handle_console_ring function in xenconsole >> xen/arm: vpl011: Add a new handle_console_tty function in xenconsole >> xen/arm: vpl011: Add a new console_cleanup function in xenconsole >> xen/arm: vpl011: Add a new console_open_log function in xenconsole >> xen/arm: vpl011: Add a new console_close_evtchn function in xenconsole >> xen/arm: vpl011: Add support for multiple consoles in xenconsole >> xen/arm: vpl011: Add support for vuart console in xenconsole >> xen/arm: vpl011: Add a new vuart console type to xenconsole client >> xen/arm: vpl011: Add a pl011 uart DT node in the guest device tree >> xen/arm: vpl011: Update documentation for vuart console support >> >> config/arm32.mk | 1 + >> config/arm64.mk | 1 + >> docs/man/xl.cfg.pod.5.in | 9 + >> docs/misc/console.txt | 44 ++- >> tools/console/Makefile | 3 +- >> tools/console/client/main.c | 13 +- >> tools/console/daemon/io.c | 662 >> ++++++++++++++++++++++++----------- >> tools/libxc/include/xc_dom.h | 2 + >> tools/libxc/include/xenctrl.h | 18 + >> tools/libxc/xc_dom_arm.c | 5 +- >> tools/libxc/xc_dom_boot.c | 2 + >> tools/libxc/xc_domain.c | 24 ++ >> tools/libxl/libxl.h | 6 + >> tools/libxl/libxl_arch.h | 6 + >> tools/libxl/libxl_arm.c | 72 +++- >> tools/libxl/libxl_console.c | 47 +++ >> tools/libxl/libxl_create.c | 9 +- >> tools/libxl/libxl_device.c | 9 +- >> tools/libxl/libxl_dom.c | 5 + >> tools/libxl/libxl_internal.h | 6 + >> tools/libxl/libxl_types.idl | 7 + >> tools/libxl/libxl_types_internal.idl | 1 + >> tools/libxl/libxl_x86.c | 8 + >> tools/xl/xl_cmdtable.c | 2 +- >> tools/xl/xl_console.c | 5 +- >> tools/xl/xl_parse.c | 8 + >> xen/arch/arm/Kconfig | 7 + >> xen/arch/arm/Makefile | 1 + >> xen/arch/arm/domain.c | 5 + >> xen/arch/arm/domctl.c | 43 ++- >> xen/arch/arm/vpl011.c | 455 ++++++++++++++++++++++++ >> xen/include/asm-arm/domain.h | 6 + >> xen/include/asm-arm/pl011-uart.h | 2 + >> xen/include/asm-arm/vpl011.h | 72 ++++ >> xen/include/public/arch-arm.h | 6 + >> xen/include/public/domctl.h | 21 ++ >> xen/include/public/io/console.h | 4 + >> 37 files changed, 1360 insertions(+), 237 deletions(-) >> create mode 100644 xen/arch/arm/vpl011.c >> create mode 100644 xen/include/asm-arm/vpl011.h >> > > -- > Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx https://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |