GNTTABOP_setup_table yields -1 PFNs

[Taylor R Campbell <riastradh-xen-devel@xxxxxxxxxx>]

[I asked on IRC and Andrew Cooper suggested posting the question here.
I'm not subscribed to xen-devel@, so please cc me in replies.]


On a Xen 4.14 host (with extraversion=.0.88.g1d1d1f53), with version 1
grant tables, where GNTTABOP_query_size initially returns nr_frames=32
max_nr_frames=64, a NetBSD guest repeatedly queries
GNTTABOP_setup_table for successively larger nr_frames from 1 up.

The guest initially gets arrays of valid-looking PFNs.  But then at
nr_frames=33, the PFNs [0] through [31] in the resulting array are
valid but PFN [32] is -1, i.e., all bits set.

GNTTABOP_setup_table returned 0 and op.status = GNTST_okay, so it
didn't fail -- it just returned an invalid PFN.  And _after_
GNTTABOP_setup_table yields -1 as a PFN, GNTTABOP_query_size returns
nr_frames=33 max_nr_frames=64, so the host thinks it has successfully
allocated more frames.

What could cause the host to return a PFN -1?  Is there anything the
guest does that could provoke this?  Are there any diagnostics that
the guest could print to help track this down?  (I don't control the
host.)  Should a guest just check for -1 and stop as if it had hit
max_nr_frames?



(NetBSD then passes the PFN -1 on to HYPERVISOR_mmu_update, which
fails with EINVAL, at which point NetBSD crashes;
<https://gnats.NetBSD.org/58395>.  If I make NetBSD pretend the host
had advertised max_nr_frames=32, everything works fine.

I'm guessing non-NetBSD guests will eventually crash when they see -1,
but NetBSD sees it at boot while maybe it takes very heavy I/O load on
Linux to make it reach nr_frames=33 in this setup.)