Re: [Xen-devel] xen-blkfront: weird behavior of "iostat" after VM live-migrate which xen-blkfront module has indirect descriptors

["Ouyang Zhaowei (Charles)" <ouyangzhaowei@xxxxxxxxxx>]

On 2015.1.26 10:30, Ouyang Zhaowei (Charles) wrote:
> 
> On 2015.1.23 19:15, Roger Pau Monnà wrote:
>> Hello,
>>
>> El 23/01/15 a les 8.59, Ouyang Zhaowei (Charles) ha escrit:
>>> Hi Roger,
>>>
>>> We are testing the indirect feature of xen-blkfront module these days.
>>> And we found that, after VM live-migrate a couple of times, the "%util" of 
>>> iostat keeps being 100%, and there are several requests stock in "avgqu-sz".
>>> We have checked some later version of Linux, and it happens on Ubuntu 
>>> 14.04, Ubuntu 14.10 and RHEL 7.0.
>>>
>>> The iostat shows like below:
>>>
>>> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>>>            0.00    0.00    0.00    0.00    0.00  100.00
>>>
>>> Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz 
>>> avgqu-sz   await r_await w_await  svctm  %util
>>> xvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00  
>>>    4.00    0.00    0.00    0.00   0.00 100.00
>>> dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00  
>>>    0.00    0.00    0.00    0.00   0.00   0.00
>>> dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00  
>>>    0.00    0.00    0.00    0.00   0.00   0.00
>>>
>>> Could you tell us that why is this happening, is this a bug?
>>
>> It is a bug indeed, thanks for reporting it. The problem seems to be 
>> that blk_put_request (which is used to discard the old requests before 
>> requeuing them) doesn't update the queue statistics. The following 
>> patch solves the problem for me, could you try it and report back?

Hi Roger,

After near 1000 times migrate test, the "%util" of iostat did not become 100% 
anymore, seems like the patch fix this bug

Thanks

Ouyang Zhaowei

>>
>> ---
>> commit bb4317c051ca81a2906edb7ccc505cbd6d1d80c7
>> Author: Roger Pau Monne <roger.pau@xxxxxxxxxx>
>> Date:   Fri Jan 23 12:10:51 2015 +0100
>>
>>     xen-blkfront: fix accounting of reqs when migrating
>>     
>>     Current migration code uses blk_put_request in order to finish a request
>>     before requeuing it. This function doesn't update the statistics of the
>>     queue, which completely screws accounting. Use blk_end_request_all 
>> instead
>>     which properly updates the statistics of the queue.
>>     
>>     Signed-off-by: Roger Pau Monnà <roger.pau@xxxxxxxxxx>
>>
>> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
>> index 5ac312f..aac41c1 100644
>> --- a/drivers/block/xen-blkfront.c
>> +++ b/drivers/block/xen-blkfront.c
>> @@ -1493,7 +1493,7 @@ static int blkif_recover(struct blkfront_info *info)
>>              merge_bio.tail = copy[i].request->biotail;
>>              bio_list_merge(&bio_list, &merge_bio);
>>              copy[i].request->bio = NULL;
>> -            blk_put_request(copy[i].request);
>> +            blk_end_request_all(copy[i].request, 0);
>>      }
>>  
>>      kfree(copy);
>> @@ -1516,7 +1516,7 @@ static int blkif_recover(struct blkfront_info *info)
>>              req->bio = NULL;
>>              if (req->cmd_flags & (REQ_FLUSH | REQ_FUA))
>>                      pr_alert("diskcache flush request found!\n");
>> -            __blk_put_request(info->rq, req);
>> +            __blk_end_request_all(req, 0);
>>      }
>>      spin_unlock_irq(&info->io_lock);
>>  
>>
> 
> Hi Roger,
> 
> Thanks for answering this question. Sure, I'll try this patch and test VM 
> migrating, so far it seems this patch has solved this bug (after 10 times 
> migrate).
> I'll keep testing it for more times and will let you know if it's OK.
> 
> Regards,
> Ouyang Zhaowei
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> .
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel