[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [xen-4.8-testing test] 124100: regressions - FAIL



On 06/13/2018 05:18 AM, Ian Jackson wrote:
Jim: please read down to where I discuss
test-amd64-amd64-libvirt-pair.  If you have any insight I'd appreciate
it.  Let me know if you want me to preserve the logs, which will
otherwise expire in a few weeks.

Whoa, sorry for the delay. This mail found a dumb bug in my filter for xen-devel mail.

  test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
123701

 From the log:

2018-06-12 20:59:40 Z executing ssh ... root@172.16.144.61 virsh migrate --live 
debian.guest.osstest xen+ssh://joubertin0
error: Timed out during operation: cannot acquire state change lock
2018-06-12 21:00:16 Z command nonzero waitstatus 256: [..]

The libvirt libxl logs seem to show libxl doing a successful
migration.

With the long delay, I'm afraid the logs have expired. Do you still see the problem? All the recent runs seem to be plagued with libvirt's change to require GnuTLS

https://libvirt.org/git/?p=libvirt.git;a=commit;h=60d9ad6f1e42618fce10baeb0f02c35e5ebd5b24

Looking at the logs I see this:

2018-06-12 21:00:16.784+0000: 3507: warning :
libxlDomainObjBeginJob:151 : Cannot start job (modify) for domain
debian.guest.osstest; current job is (modify) owned by (24947)

That job number looks like it's about right for a pid, but I think it
must be a thread because it doesn't show up in the ps output.

Likely a libvirtd worker thread doing something that requires modifying the state of virDomainObj.

I did see this:

Jun 12 21:00:20 joubertin0 logger: /etc/xen/scripts/vif-bridge: iptables setup 
failed. This may affect guest networking.

but that seems to be after the failure.

A wild guess, but is it possible thread 24947 is running a domain create operation, which includes executing vif-bridge, that is taking longer than expected to complete?

I don't have an explanation.  I don't really know what this lock is.

It's a lock that serializes domain state modifications (changing virDomainObj). Wait time for the lock is currently hardcoded to 30sec. The thread emitting the warning surpassed the timeout, waiting for 24947 to finish whatever it was doing.

Regards,
Jim

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.