[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] libvirtd hang on CentOS6 after latest updates



On Wed, Jul 25, 2018 at 4:42 PM, Jim Fehlig <jfehlig@xxxxxxxx> wrote:
> On 07/22/2018 04:03 PM, Karel Hendrych wrote:
>>
>> Hi, I am seeing frequent libvirtd hangs (clients not responding) after
>> last CentOS6-Xen update :
>
>
> xen-devel is not the best place to seek help with downstream issues,
> particularly libvirt ones :-). You would have better luck contacting the
> CentOS6 maintainers.

In this case, it looks very much like they're suing the Virt SIG
binaries, which are pretty close to being straight-up packing of the
upstream tarballs, and the maintainers would be Anthony & I.  And I at
least know very little about libvirt.  If Karel had posted this on
centos-devel, I would almost certainly have ended up asking him to
repost to xen-devel anyway, at which point I would have cc'd you. :-)

Does the error ring any bells?

Karel, is there a reason you're still on Xen 4.6 rather than 4.8?  I
don't think it's getting security updates at this point.  (Anthony & I
are only committing to maintaining a single Xen version, and right now
that's 4.8.)

 -George

>
>>
>> libvirt-libs-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-network-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-nwfilter-4.1.0-2.xen46.el6.x86_64
>> libgcc-4.4.7-18.el6_9.2.x86_64
>> 2:qemu-img-0.12.1.2-2.503.el6_9.5.x86_64
>> libvirt-daemon-driver-storage-core-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-secret-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-interface-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-nodedev-4.1.0-2.xen46.el6.x86_64
>> 10:centos-release-xen-common-8-4.el6.x86_64
>> xen-licenses-4.6.6-12.el6.x86_64
>> xen-libs-4.6.6-12.el6.x86_64
>> libvirt-daemon-driver-libxl-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-xen-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-qemu-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-gluster-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-logical-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-mpath-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-disk-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-scsi-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-iscsi-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-storage-4.1.0-2.xen46.el6.x86_64
>> libstdc++-4.4.7-18.el6_9.2.x86_64
>> libvirt-daemon-config-nwfilter-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-config-network-4.1.0-2.xen46.el6.x86_64
>> libvirt-daemon-driver-lxc-4.1.0-2.xen46.el6.x86_64
>> libvirt-client-4.1.0-2.xen46.el6.x86_64
>> linux-firmware-20171215-82.git2451bb22.el6.noarch
>> 12:dhcp-common-4.1.1-53.P1.el6.centos.4.x86_64
>> 12:dhclient-4.1.1-53.P1.el6.centos.4.x86_64
>> libvirt-4.1.0-2.xen46.el6.x86_64
>> 10:centos-release-xen-46-8-4.el6.x86_64
>> 10:centos-release-xen-44-8-4.el6.x86_64
>> tzdata-2018e-3.el6.noarch
>> libgomp-4.4.7-18.el6_9.2.x86_64
>> kernel-4.9.86-30.el6.x86_64
>> xen-hypervisor-4.6.6-12.el6.x86_64
>> xen-runtime-4.6.6-12.el6.x86_64
>> xen-4.6.6-12.el6.x86_64
>> libvirt-daemon-xen-4.1.0-2.xen46.el6.x86_64
>>
>> Remedy is to kill -9 libvirtd and start again. Issue can be replicated
>> within few domU starts. Usually libvirtd hangs when domU is bringing up xen
>> drivers or something around udev, like:
>>
>> xen_netfront: Initialising Xen virtual ethernet driver
>>
>> I've been looking into libvirtd strace and debug logs, so far most
>> suspicious in libvirtd debug log is this:
>>
>> libvirtd.log:2018-05-22 08:32:44.760+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-7'
>> libvirtd.log:2018-05-22 08:32:44.761+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-6'
>> libvirtd.log:2018-05-22 08:32:44.761+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-4'
>> libvirtd.log:2018-05-22 08:32:44.762+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-5'
>> libvirtd.log:2018-05-22 08:32:44.763+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-2'
>> libvirtd.log:2018-05-22 08:32:44.764+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/tx-3'
>> libvirtd.log:2018-05-22 08:32:44.765+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-6'
>> libvirtd.log:2018-05-22 08:32:44.766+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-5'
>> libvirtd.log:2018-05-22 08:32:44.767+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-4'
>> libvirtd.log:2018-05-22 08:32:44.767+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-7'
>> libvirtd.log:2018-05-22 08:32:44.768+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-2'
>> libvirtd.log:2018-05-22 08:32:44.769+0000: 25455: debug :
>> udevRemoveOneDevice:1289 : Failed to find device to remove that has udev
>> name '/sys/devices/vif-24-0/net/vif24.0/queues/rx-3'
>>
>> I could not get rid of this by reducing amount of driver queues (not sure
>> if that applies to PV)
>>
>> Is someone out there seeing similar issues? Anyone perhaps interested in
>> reviewing full debug log / strace ?
>>
>> Cheers
>> Karel
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxxx
>> https://lists.xenproject.org/mailman/listinfo/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxxx
> https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.