[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v6 1/5] x86/mem_sharing: reorder when pages are unlocked and released


  • To: Tamas K Lengyel <tamas@xxxxxxxxxxxxx>
  • From: Jan Beulich <JBeulich@xxxxxxxx>
  • Date: Thu, 18 Jul 2019 14:28:18 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a+kBLf8I0/5WQRoaEKDKffxPZOvI3M9u2+BoZoszg/A=; b=VkHDxnVzZLvdyyptha1+SDInnr19xwFV4eRVHVZ474kuklgB7E4MoZuJS4xJOsxMmAe/4FMUMgFjdTkFD9gGSFFfA86RLxGN17F8QW/pa828GomoG8UxbWqn0CDDgWooyR8RbQRHcvBj/DoWnd38vrhLfluWp8IZsrtX5+1hsWSUYyWe9394dGmkS5FfZy21cYfg6132OIKhc/I8QjbtJtyV0wDpIkN7A1hypUJ6SnEr4tCmVU5DPNstyWIt7F1qCNbAKVAPF+S5YTuusurQsWIqQqH18myHMGBEJlrxeX2k4E0VOTWRbeSRr+fjI0LLQ59H/BmjZVHpY5jpd+GaGg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GcZ2LtxdlzICa7MUt2dogKYfYq1WgClIJoQmbNWnX2WO62XW2AZBFHlsuP/WAs0b2KNulEvoTeHHBFr/zkICDvJnzNhY6Zb81ysGD/tU0NIvGeSrCq1O1wm9LVVfqYJu/OEfYFkBagb+hzPlQYC829Z8eXaf+Hcmvlq1K1j+5mwaJ6S6Y7CfQrtxzEPO4iIH9g/kQMpPAwvxgfxfb+pLDCX2U604Jbv56KDt8ff3dkpSca6EYtx2X/ywXX5zYZmA/vNUDRzPsMVSOhM2NGGxRoeTGAT4fFKnP+H3vsZ+JP+KWQD6C7M7wm0NnooJYvioEL6y7kjIZc3kGOzJtr/0sw==
  • Authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@xxxxxxxx;
  • Cc: George Dunlap <george.dunlap@xxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Wei Liu <wei.liu2@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Roger Pau Monne <roger.pau@xxxxxxxxxx>
  • Delivery-date: Thu, 18 Jul 2019 14:28:46 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHVPNaoMN4/6VqHlEm91bwrx18tx6bQMluAgAAkhnuAAARMgIAAAL3BgAAFHgCAAAROh4AACw0A
  • Thread-topic: [Xen-devel] [PATCH v6 1/5] x86/mem_sharing: reorder when pages are unlocked and released

On 18.07.2019 15:47, Tamas K Lengyel wrote:
> On Thu, Jul 18, 2019 at 7:33 AM Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>
>> On 18.07.2019 15:13, Tamas K Lengyel wrote:
>>> On Thu, Jul 18, 2019 at 7:12 AM Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>
>>>> On 18.07.2019 14:55, Tamas K Lengyel wrote:
>>>>> On Thu, Jul 18, 2019 at 4:47 AM Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>>> On 17.07.2019 21:33, Tamas K Lengyel wrote:
>>>>>>> @@ -900,6 +895,7 @@ static int share_pages(struct domain *sd, gfn_t 
>>>>>>> sgfn, shr_handle_t sh,
>>>>>>>          p2m_type_t smfn_type, cmfn_type;
>>>>>>>          struct two_gfns tg;
>>>>>>>          struct rmap_iterator ri;
>>>>>>> +    unsigned long put_count = 0;
>>>>>>>
>>>>>>>          get_two_gfns(sd, sgfn, &smfn_type, NULL, &smfn,
>>>>>>>                       cd, cgfn, &cmfn_type, NULL, &cmfn, 0, &tg);
>>>>>>> @@ -964,15 +960,6 @@ static int share_pages(struct domain *sd, gfn_t 
>>>>>>> sgfn, shr_handle_t sh,
>>>>>>>              goto err_out;
>>>>>>>          }
>>>>>>>
>>>>>>> -    /* Acquire an extra reference, for the freeing below to be safe. */
>>>>>>> -    if ( !get_page(cpage, dom_cow) )
>>>>>>> -    {
>>>>>>> -        ret = -EOVERFLOW;
>>>>>>> -        mem_sharing_page_unlock(secondpg);
>>>>>>> -        mem_sharing_page_unlock(firstpg);
>>>>>>> -        goto err_out;
>>>>>>> -    }
>>>>>>> -
>>>>>>>          /* Merge the lists together */
>>>>>>>          rmap_seed_iterator(cpage, &ri);
>>>>>>>          while ( (gfn = rmap_iterate(cpage, &ri)) != NULL)
>>>>>>> @@ -984,13 +971,14 @@ static int share_pages(struct domain *sd, gfn_t 
>>>>>>> sgfn, shr_handle_t sh,
>>>>>>>               * Don't change the type of rmap for the client page. */
>>>>>>>              rmap_del(gfn, cpage, 0);
>>>>>>>              rmap_add(gfn, spage);
>>>>>>> -        put_page_and_type(cpage);
>>>>>>> +        put_count++;
>>>>>>>              d = get_domain_by_id(gfn->domain);
>>>>>>>              BUG_ON(!d);
>>>>>>>              BUG_ON(set_shared_p2m_entry(d, gfn->gfn, smfn));
>>>>>>>              put_domain(d);
>>>>>>>          }
>>>>>>>          ASSERT(list_empty(&cpage->sharing->gfns));
>>>>>>> +    BUG_ON(!put_count);
>>>>>>>
>>>>>>>          /* Clear the rest of the shared state */
>>>>>>>          page_sharing_dispose(cpage);
>>>>>>> @@ -1001,7 +989,9 @@ static int share_pages(struct domain *sd, gfn_t 
>>>>>>> sgfn, shr_handle_t sh,
>>>>>>>
>>>>>>>          /* Free the client page */
>>>>>>>          put_page_alloc_ref(cpage);
>>>>>>> -    put_page(cpage);
>>>>>>> +
>>>>>>> +    while ( put_count-- )
>>>>>>> +        put_page_and_type(cpage);
>>>>>>>
>>>>>>>          /* We managed to free a domain page. */
>>>>>>>          atomic_dec(&nr_shared_mfns);
>>>>>>> @@ -1165,19 +1155,13 @@ int __mem_sharing_unshare_page(struct domain *d,
>>>>>>>          {
>>>>>>>              if ( !last_gfn )
>>>>>>>                  mem_sharing_gfn_destroy(page, d, gfn_info);
>>>>>>> -        put_page_and_type(page);
>>>>>>> +
>>>>>>>              mem_sharing_page_unlock(page);
>>>>>>> +
>>>>>>>              if ( last_gfn )
>>>>>>> -        {
>>>>>>> -            if ( !get_page(page, dom_cow) )
>>>>>>> -            {
>>>>>>> -                put_gfn(d, gfn);
>>>>>>> -                domain_crash(d);
>>>>>>> -                return -EOVERFLOW;
>>>>>>> -            }
>>>>>>>                  put_page_alloc_ref(page);
>>>>>>> -            put_page(page);
>>>>>>> -        }
>>>>>>> +
>>>>>>> +        put_page_and_type(page);
>>>>>>>              put_gfn(d, gfn);
>>>>>>>
>>>>>>>              return 0;
>>>>>>
>>>>>> ... this (main, as I guess by the title) part of the change? I think
>>>>>> you want to explain what was wrong here and/or why the new arrangement
>>>>>> is better. (I'm sorry, I guess it was this way on prior versions
>>>>>> already, but apparently I didn't notice.)
>>>>>
>>>>> It's what the patch message says - calling put_page_and_type before
>>>>> mem_sharing_page_unlock can cause a deadlock. Since now we are now
>>>>> holding a reference to the page till the end there is no need for the
>>>>> extra get_page/put_page logic when we are dealing with the last_gfn.
>>>>
>>>> The title says "reorder" without any "why".
>>>
>>> Yes, I can't reasonably fit "Calling _put_page_type while also holding
>>> the page_lock for that page can cause a deadlock." into the title. So
>>> it's spelled out in the patch message.
>>
>> Of course not. And I realize _part_ of the changes is indeed what the
>> title says (although for share_pages() that's not obvious from the
>> patch alone). But you do more: You also avoid acquiring an extra
>> reference in share_pages().
> 
> I feel like we are going in circles and having the same conversations
> over and over without really making any headway. You introduced
> grabbing the broken extra reference in 0502e0adae2. It is and was
> actually unneeded to start with if the proper solution was put in
> place, which is what this patch does, reordering things.

I'm not complaining about the changes; I'd merely like the description
state why they're needed.

>> And since you made me look at the code again: If put_page() is unsafe
>> with a lock held, how come the get_page_and_type() in share_pages()
>> is safe with two such locks held? If it really is, it surely would be
>> worthwhile to state in the description. There's another such instance
>> in mem_sharing_add_to_physmap() (plus a get_page()), and also one
>> where put_page_and_type() gets called with such a lock held (afaics).
> 
> It's possible there are other instances where this may still be
> broken. Right now I only have bandwidth to test and fix the paths I
> use. If that's unacceptable I'm happy to continue development in my
> private fork and leave things as-is upstream.

Similarly, if there are limitations - fine. But please say so in the
description, to avoid giving the impression that the issues have been
taken care of altogether.

Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.