[Xen-devel] [PATCH] qemu-dm drops into infinite loop

[Kouya SHIMURA <kouya@xxxxxxxxxxxxxx>]

Hi,

I encountered the problem that qemu-dm hangs up.
After some investigation, I found qemu-dm drops into infinite loop
in function vnc_client_read() of tools/ioemu/vnc.c.

    while (vs->read_handler && vs->input.offset >= vs->read_handler_expect) {
        size_t len = vs->read_handler_expect;
        int ret;

        ret = vs->read_handler(vs, vs->input.buffer, len);
        ...
        if (!ret) {
            ...
        } else
            vs->read_handler_expect = ret;
    }

Attached patch prevents qemu-dm from causing the infinite loop.

Thanks
Kouya

Signed-off-by: Kouya Shimura <kouya@xxxxxxxxxxxxxx>

diff -r 7acaba46e15e tools/ioemu/vnc.c
--- a/tools/ioemu/vnc.c Fri Sep 08 15:46:54 2006 -0700
+++ b/tools/ioemu/vnc.c Mon Sep 11 19:28:31 2006 +0900
@@ -1016,6 +1016,7 @@ static int protocol_client_msg(VncState 
 {
     int i;
     uint16_t limit;
+    size_t cut_len;
 
     switch (data[0]) {
     case 0:
@@ -1032,10 +1033,11 @@ static int protocol_client_msg(VncState 
        if (len == 1)
            return 4;
 
-       if (len == 4)
-           return 4 + (read_u16(data, 2) * 4);
-
        limit = read_u16(data, 2);
+
+       if (len == 4 && limit > 0)
+           return 4 + (limit * 4);
+
        for (i = 0; i < limit; i++) {
            int32_t val = read_s32(data, 4 + (i * 4));
            memcpy(data + 4 + (i * 4), &val, sizeof(val));
@@ -1067,10 +1069,12 @@ static int protocol_client_msg(VncState 
        if (len == 1)
            return 8;
 
-       if (len == 8)
-           return 8 + read_u32(data, 4);
-
-       client_cut_text(vs, read_u32(data, 4), data + 8);
+       cut_len = read_u32(data, 4);
+
+       if (len == 8 && cut_len > 0)
+           return 8 + cut_len;
+
+       client_cut_text(vs, cut_len, data + 8);
        break;
     default:
        printf("Msg: %d\n", data[0]);

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel