Domain reference counting breakage

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

At the moment, attempting to create an HVM guest with max_gnttab_frames of 0
causes Xen to explode on the:

  BUG_ON(atomic_read(&d->refcnt) != DOMAIN_DESTROYED);

in _domain_destroy().  Intrumenting Xen a little more to highlight where the
modifcations to d->refcnt occur:

  (d6) --- Xen Test Framework ---
  (d6) Environment: PV 64bit (Long mode 4 levels)
  (d6) Testing domain create:
  (d6) Testing x86 PVH Shadow
  (d6) (XEN) d0v0 Hit #DB in Xen context: e008:ffff82d0402046b5 
[domain_create+0x1c3/0x7f1], stk e010:ffff83003fea7d58, dr6 ffff0ff1
  (d6) (XEN) d0v0 Hit #DB in Xen context: e008:ffff82d040321b11 
[share_xen_page_with_guest+0x175/0x190], stk e010:ffff83003fea7ce8, dr6 ffff0ff1
  (d6) (XEN) d0v0 Hit #DB in Xen context: e008:ffff82d04022595b 
[assign_pages+0x223/0x2b7], stk e010:ffff83003fea7c68, dr6 ffff0ff1
  (d6) (XEN) grant_table.c:1934: Bad grant table sizes: grant 0, maptrack 0
  (d6) (XEN) *** d1 ref 3
  (d6) (XEN) d0v0 Hit #DB in Xen context: e008:ffff82d0402048bc 
[domain_create+0x3ca/0x7f1], stk e010:ffff83003fea7d58, dr6 ffff0ff1
  (d6) (XEN) d0v0 Hit #DB in Xen context: e008:ffff82d040225e11 
[free_domheap_pages+0x422/0x44a], stk e010:ffff83003fea7c38, dr6 ffff0ff1
  (d6) (XEN) Xen BUG at domain.c:450
  (d6) (XEN) ----[ Xen-4.15-unstable  x86_64  debug=y  Not tainted ]----
  (d6) (XEN) CPU:    0
  (d6) (XEN) RIP:    e008:[<ffff82d040204366>] 
common/domain.c#_domain_destroy+0x69/0x6b

the problem becomes apparent.

First of all, there is a reference count leak - share_xen_page_with_guest()'s
reference isn't freed anywhere.

However, the main problem is the 4th #DB above is this atomic_set()

  d->is_dying = DOMDYING_dead;
  if ( hardware_domain == d )
      hardware_domain = old_hwdom;
  printk("*** %pd ref %d\n", d, atomic_read(&d->refcnt));
  atomic_set(&d->refcnt, DOMAIN_DESTROYED);

in the domain_create() error path, which happens before free_domheap_pages()
drops the ref acquired assign_pages(), and destroys still-relevant information
pertaining to the guest.

The best options is probably to use atomic_sub() to subtract (DOMAIN_DESTROYED
+ 1) from the current refcount, which preserves the extra refs taken by
share_xen_page_with_guest() and assign_pages() until they can be freed
appropriately.

~Andrew