Re: [PATCH DO NOT APPLY] docs: Document allocator properties and the rubric for using them

[Jan Beulich <jbeulich@xxxxxxxx>]

On 16.02.2021 11:28, George Dunlap wrote:
> --- /dev/null
> +++ b/docs/hypervisor-guide/memory-allocation-functions.rst
> @@ -0,0 +1,118 @@
> +.. SPDX-License-Identifier: CC-BY-4.0
> +
> +Xenheap memory allocation functions
> +===================================
> +
> +In general Xen contains two pools (or "heaps") of memory: the *xen
> +heap* and the *dom heap*.  Please see the comment at the top of
> +``xen/common/page_alloc.c`` for the canonical explanation.
> +
> +This document describes the various functions available to allocate
> +memory from the xen heap: their properties and rules for when they should be
> +used.

Irrespective of your subsequent indication of you disliking the
proposal (which I understand only affects the guidelines further
down anyway) I'd like to point out that vmalloc() does not
allocate from the Xen heap. Therefore a benefit of always
recommending use of xvmalloc() would be that the function could
fall back to vmalloc() (and hence the larger domain heap) when
xmalloc() failed.

> +TLDR guidelines
> +---------------
> +
> +* By default, ``xvmalloc`` (or its helper cognates) should be used
> +  unless you know you have specific properties that need to be met.
> +
> +* If you need memory which needs to be physically contiguous, and may
> +  be larger than ``PAGE_SIZE``...
> +  
> +  - ...and is order 2, use ``alloc_xenheap_pages``.
> +    
> +  - ...and is not order 2, use ``xmalloc`` (or its helper cognates)..

ITYM "an exact power of 2 number of pages"?

> +* If you don't need memory to be physically contiguous, and know the
> +  allocation will always be larger than ``PAGE_SIZE``, you may use
> +  ``vmalloc`` (or one of its helper cognates).
> +
> +* If you know that allocation will always be less than ``PAGE_SIZE``,
> +  you may use ``xmalloc``.

As per Julien's and your own replies, this wants to be "minimum
possible page size", which of course depends on where in the
tree the piece of code is to live. (It would be "maximum
possible page size" in the earlier paragraph.)

> +Properties of various allocation functions
> +------------------------------------------
> +
> +Ultimately, the underlying allocator for all of these functions is
> +``alloc_xenheap_pages``.  They differ on several different properties:
> +
> +1. What underlying allocation sizes are.  This in turn has an effect
> +   on:
> +
> +   - How much memory is wasted when requested size doesn't match
> +
> +   - How such allocations are affected by memory fragmentation
> +
> +   - How such allocations affect memory fragmentation
> +
> +2. Whether the underlying pages are physically contiguous
> +
> +3. Whether allocation and deallocation require the cost of mapping and
> +   unmapping
> +
> +``alloc_xenheap_pages`` will allocate a physically contiguous set of
> +pages on orders of 2.  No mapping or unmapping is done.

That's the case today, but meant to change rather sooner than later
(when the 1:1 map disappears).

Jan