[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Possible bug with huge unflushed console buffer


  • To: Mark van Dijk <lists+xen@xxxxxxxxxxxxxx>, <xen-devel@xxxxxxxxxxxxx>
  • From: Keir Fraser <keir@xxxxxxx>
  • Date: Wed, 08 Aug 2012 17:42:23 +0100
  • Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
  • Delivery-date: Wed, 08 Aug 2012 16:42:41 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xen.org>
  • Thread-index: Ac11hMjiMN8F120MA0CVQDUc3XZfaQ==
  • Thread-topic: [Xen-devel] Possible bug with huge unflushed console buffer

On 08/08/2012 16:43, "Mark van Dijk" <lists+xen@xxxxxxxxxxxxxx> wrote:

> Greetings list,
> 
> Yesterday the following scenario took place. In short, I had a HVM host
> with a poorly configured firewall. It logged way more than it should,
> and it had been doing that for days. IPTables entries with -j LOG
> show up in the console. I told Konrad about this but I said it was a PV
> host while in fact it is concerning a HVM host.
> 
> Anyway, when I saw that this was the case I corrected iptables and then
> I removed those faulty log entries from the logfiles in /var/log.
> Finally I issued /etc/init.d/rsyslogd reload.
> 
> Suddenly the server stopped responding at all, not even arp requests
> were answered. I opened up a console via xl (none was open before
> this) and I was greeted with a crazy amount of log entries - it took
> about five or six minutes before the whole buffer was flushed to my
> screen. Then the server responded again.
> 
> Today I viewed syslog's messages in detail and found this and a lot of
> similar entries:
> 
> http://pastebin.com/Ga7aE7hb
> 
> So the questions that I have now are - Is this a bug? How is console
> history handled and where is the console's unread history stored?
> Somewhere in the hypervisor or in dom0 or domU memory space? Is there a
> limit to how much info can be stored?
> 
> The behaviour I encountered i.e. a system lock seems to suggest that
> there was some kind of buffer overflow. And this can be achieved by
> something as simple as 'iptables -I INPUT -j LOG'. Perhaps it is a wise
> idea to consider xl console -c to clear the console's history, then at
> least I could exit the console and clear unsent messages...

Is your VM logging to its virtual serial line? You could just not do that...

I think this is due to the guest's qemu-dm process stuffing its transmitted
serial data into a pty, to be consumed by 'xl console', and if that doesn't
happen the buffers fill up, serial processing stops, and we back up all the
way to the guest.

 -- Keir

> I am not receiving xen-devel messages, so please CC me on replies.
> Thank you.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.