WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: Linux Stubdom Problem

To: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
Subject: [Xen-devel] Re: Linux Stubdom Problem
From: Jiageng Yu <yujiageng734@xxxxxxxxx>
Date: Mon, 22 Aug 2011 23:24:52 +0800
Cc: Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxx>, "Xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Samuel Thibault <samuel.thibault@xxxxxxxxxxxx>
Delivery-date: Mon, 22 Aug 2011 08:26:11 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=+uGYknYA9l2MXzjQwxD/sqPTsUNFqDti5dVpLc98pzI=; b=NG4fbdZhg4N1O4kZ+FrO3G+J+4qHM53l2b+bLuaHKXwBY1O4ynv9aIGvr7BbzpA5RL CGYfykKT02fq5lutRXRs0KXX/FgVFRFgdx1P322K3nwbYZRvOir6sAt+ER7LHQHJvpBU 6iUCLwv8KU9RkyHkOErkpfA2bBNakFFzHhsOU=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <alpine.DEB.2.00.1108190027450.12963@kaball-desktop>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <CAJ0pt14RBmT+bCGhU6szMW4Aje-mBQ-WVR8Vb7wOLefgauatbA@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107271224350.12963@kaball-desktop> <CAJ0pt15tkb8F6LNHxSwjVmCF9DvvJjZqQKU-TXKyqT_seZibmw@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107271431070.12963@kaball-desktop> <CAJ0pt15-X7psh5Fzxzo0=8BR9G-hdVjdPqQO7CYLDCgNx9zNZg@xxxxxxxxxxxxxx> <CAJ0pt175GrngJTxnvx0GRf1RvmXe_JAMxBch+SprKOArNx42ng@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107281928020.12963@kaball-desktop> <CAJ0pt146Te2hCFpxCdwU518HpwRuGHiUj7sSkVOU75YLpoXSBw@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107291601070.12963@kaball-desktop> <CAJ0pt15t_7==0rX9qEHyDMBRWUgfpMyLCmP0yBJsZg410FoutA@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107291617130.12963@kaball-desktop> <CAJ0pt15MVvw5_rJ_xFxR6aQpRYtTWFmR0OHJ54wuXSE+7hNkGQ@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107291628000.12963@kaball-desktop> <CAJ0pt155eqcyUwd-m--m3n+t-hLYixe+xA8J5MWgEWj58+o_Gw@xxxxxxxxxxxxxx> <CAJ0pt14OXshWn5NQ3JSfqRNdHkWMyhKWC0OwMixu9Xmk0b1oqw@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1108190027450.12963@kaball-desktop>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
2011/8/19 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
> On Mon, 15 Aug 2011, Jiageng Yu wrote:
>> Hi Stefano,
>>
>>     In the linux-pv xenfbfront driver, the vram is allocated by:
>>     xenfb_probe()
>>         ->info->fb = vmalloc(fb_size);
>>
>>     In the linux-stubdom, the memory areas: (info->fb,
>> info->fb+fb_size) in linux-pv kernel, (s->vram_offset,
>> s->vram_offset+VGA_RAM_SIZE) in vga_common_init function,
>> (s->vram_ptr, s->vram_ptr+VGA_RAM_SIZE) in stubdom. These memory areas
>> should be mapped into the same machine memory region.
>>
>>     But the (info->fb, info->fb+fb_size) in linux-pv kernel is
>> allocated independently. I have two optional plans to slove the
>> problem.
>>
>> 1. Modify linux-pv kernel.
>>     This plan is more closer to the minios stubdom. First, I delay the
>> initial process of xenfbfront driver until qemu allocates s->vram_ptr.
>> Then, I set info->fb = s->vram_ptr.
>>
>> 2. Modify libxc.
>>     The s->vram_ptr is generated by mmap() function in
>> linux_privcmd_map_foreign_bulk(). Maybe I could replace the return
>> value of mmap() with the info->fb, which is obtained from xenstore.
>
> you would also need to modify qemu_ram_alloc to make sure the same pages
> are remapped into the guest as videoram
>
>
>>     Can these plans solve the problem? Or there is the better one?
>
> I think it is best to keep the memory allocation in xenfbfront as it is
> and find a way to map the pages in qemu. Isn't possible to use a mmap on
> the fbdev device to obtain a mapping of the vmalloc'ed area into qemu?
> Once we have mapped the pages into qemu address space it is just a
> matter of using them as videoram.  Otherwise you would have timing
> issues because qemu is probably going to start after xenfbfront has
> already allocated the buffer and enstablished a connection with the
> backend.
>
> In any case why don't you try running a guest without a video card (the
> qemu command line option for that is -vga none or -nographic) and see if
> everything else works?
>

Hi Stefano,

     I am trying to fix the vram mapping issue. That is my new patch.
It is still needed to debug. Please check it for me and make sure I am
running on the right way.

    I define a new enmu type Stubdom_Vga_State which is used to notify
xen_remap_bucket whether to map the vram memory. In
fbdev_pv_display_prepare function, I map the xen_fbfront memory to
qemu (fb_mem) and directly incoke ioctl(fd, IOCTL_PRIVCMD_MMAPBATCH,
&ioctlx) to map the vram of hvm guest.


diff --git a/fbdev.c b/fbdev.c
index a601b48..f7ff682 100644
--- a/fbdev.c
+++ b/fbdev.c
@@ -30,6 +30,12 @@
 #include "sysemu.h"
 #include "pflib.h"

+#include "hw/xen_backend.h"
+#include <xen/hvm/params.h>
+#include <sys/ioctl.h>
+#include "xenctrlosdep.h"
+#include <xen/privcmd.h>
+
 /* -------------------------------------------------------------------- */

 /* file handles */
@@ -541,29 +547,23 @@ static void fbdev_cleanup(void)
     }
 }

-static int fbdev_init(const char *device)
+static int fbdev_init(int prot, unsigned long size)
 {
     struct vt_stat vts;
        unsigned long page_mask;
     char ttyname[32];

     /* open framebuffer */
-    if (device == NULL) {
-       device = getenv("FRAMEBUFFER");
-    }
-    if (device == NULL) {
-       device = "/dev/fb0";
-    }
-    fb = open(device, O_RDWR);
+    fb = open("/dev/fb0", O_RDWR);
     if (fb == -1) {
-       fprintf(stderr, "open %s: %s\n", device, strerror(errno));
+        fprintf(stderr, "open /dev/fb0: %s\n", strerror(errno));
         return -1;
     }

     /* open virtual console */
     tty = 0;
     if (ioctl(tty, VT_GETSTATE, &vts) < 0) {
             fprintf(stderr, "Not started from virtual terminal, trying to
open one.\n");

         snprintf(ttyname, sizeof(ttyname), "/dev/tty0");
         tty = open(ttyname, O_RDWR);
@@ -632,14 +632,14 @@ static int fbdev_init(const char *device)
        goto err;
     }

     page_mask = getpagesize()-1;
     fb_mem_offset = (unsigned long)(fb_fix.smem_start) & page_mask;
-    fb_mem = mmap(NULL,fb_fix.smem_len+fb_mem_offset,
-                 PROT_READ|PROT_WRITE,MAP_SHARED,fb,0);
+    fb_mem = mmap(NULL, size << XC_PAGE_SHIFT, prot, MAP_SHARED, fb, 0);
     if (fb_mem == MAP_FAILED) {
        perror("mmap");
        goto err;
     }
+
     /* move viewport to upper left corner */
     if (fb_var.xoffset != 0 || fb_var.yoffset != 0) {
        fb_var.xoffset = 0;
@@ -930,3 +928,71 @@ void fbdev_display_uninit(void)
     qemu_remove_exit_notifier(&exit_notifier);
     uninit_mouse();
 }
+
+int fbdev_pv_display_prepare(unsigned long domid, int prot, const
unsigned long *arr,
+                                                               int *err, 
unsigned int num)
+{
+       xen_pfn_t *pfn;
+       privcmd_mmapbatch_t ioctlx;
+       int fd,i,rc;
+
+    if (fbdev_init(prot,num) != 0) {
+        exit(1);
+    }
+
+       pfn = malloc(num * sizeof(*pfn));
+       if(!pfn){
+               errno = -ENOMEM;
+               rc = -1;
+               return rc;
+       }
+       memcpy(pfn, arr, num*sizeof(*arr));
+
+       fd = open("/proc/xen/privcmd", O_RDWR);
+       if(fd == -1){
+               fprintf(stderr,"Could not obtain handle on privcmd device\n");
+               exit(1);
+       }
+
+       ioctlx.num = num;
+       ioctlx.dom = domid;
+       ioctlx.addr = (unsigned long)fb_mem;
+       ioctlx.arr = pfn;
+
+       rc = ioctl(fd, IOCTL_PRIVCMD_MMAPBATCH, &ioctlx);
+
+       for(i=0; i<num; i++)
+       {
+               switch(pfn[i] ^ arr[i])
+               {
+                       case 0:
+                               err[i] = rc != -ENOENT ? rc:0;
+                               continue;
+                       default:
+                               err[i] = -EINVAL;
+                               continue;
+               }
+               break;
+       }
+
+       free(pfn);
+
+       if (rc == -ENOENT && i == num)
+               rc=0;
+       else if(rc)
+       {
+               errno = -rc;
+               rc = -1;
+       }
+
+       if(rc < 0)
+       {
+               fprintf(stderr,"privcmd ioctl failed\n");
+               munmap(fb_mem, num << XC_PAGE_SHIFT);
+               return NULL;
+       }
+
+       close(fd);
+
+       return fb_mem;
+}
diff --git a/hw/vga.c b/hw/vga.c
index 0f54734..de7dd85 100644
--- a/hw/vga.c
+++ b/hw/vga.c
@@ -28,6 +28,7 @@
 #include "vga_int.h"
 #include "pixel_ops.h"
 #include "qemu-timer.h"
+#include "xen_backend.h"

 //#define DEBUG_VGA
 //#define DEBUG_VGA_MEM
@@ -2237,7 +2238,12 @@ void vga_common_init(VGACommonState *s, int vga_ram_size)
     s->is_vbe_vmstate = 0;
 #endif
     s->vram_offset = qemu_ram_alloc(NULL, "vga.vram", vga_ram_size);
+#ifdef CONFIG_STUBDOM
+       stubdom_vga_state = VGA_INIT_READY;
+#endif
     s->vram_ptr = qemu_get_ram_ptr(s->vram_offset);
     s->vram_size = vga_ram_size;
     s->get_bpp = vga_get_bpp;
     s->get_offsets = vga_get_offsets;
diff --git a/hw/xen_backend.c b/hw/xen_backend.c
index c506dfe..f4ecce4 100644
--- a/hw/xen_backend.c
+++ b/hw/xen_backend.c
@@ -48,6 +48,10 @@ XenGnttab xen_xcg = XC_HANDLER_INITIAL_VALUE;
 struct xs_handle *xenstore = NULL;
 const char *xen_protocol;

+#ifdef CONFIG_STUBDOM
+enum Stubdom_Vga_State stubdom_vga_state=0;
+#endif
+
 /* private */
 static QTAILQ_HEAD(XenDeviceHead, XenDevice) xendevs =
QTAILQ_HEAD_INITIALIZER(xendevs);
 static int debug = 0;
diff --git a/hw/xen_backend.h b/hw/xen_backend.h
index 3305630..36167d1 100644
--- a/hw/xen_backend.h
+++ b/hw/xen_backend.h
@@ -60,6 +60,16 @@ extern XenXC xen_xc;
 extern struct xs_handle *xenstore;
 extern const char *xen_protocol;

+#ifdef CONFIG_STUBDOM
+/* linux stubdom vga initialization*/
+enum Stubdom_Vga_State{
+       VGA_INIT_WAIT = 0,
+       VGA_INIT_READY,
+       VGA_INIT_COMPLETE
+};
+extern enum Stubdom_Vga_State stubdom_vga_state;
+#endif
+
 /* xenstore helper functions */
 int xenstore_write_str(const char *base, const char *node, const char *val);
 int xenstore_write_int(const char *base, const char *node, int ival);
diff --git a/xen-mapcache.c b/xen-mapcache.c
index 007136a..e615f98 100644
--- a/xen-mapcache.c
+++ b/xen-mapcache.c
@@ -20,6 +20,7 @@
 #include "xen-mapcache.h"
 #include "trace.h"

+#include "hw/xen.h"

 //#define MAPCACHE_DEBUG

@@ -144,8 +145,19 @@ static void xen_remap_bucket(MapCacheEntry *entry,
         pfns[i] = (address_index << (MCACHE_BUCKET_SHIFT-XC_PAGE_SHIFT)) + i;
     }

+#ifdef CONFIG_STUBDOM
+       if(stubdom_vga_state == VGA_INIT_READY){
+               fprintf(stderr,"xen_remap_bucket: start linux stubdom vga\n");
+               vaddr_base = fbdev_pv_display_prepare(xen_domid, 
PROT_READ|PROT_WRITE,
+                                                                               
pfns, err, nb_pfn);
+               stubdom_vga_state = VGA_INIT_COMPLETE;
+       }else
+       vaddr_base = xc_map_foreign_bulk(xen_xc, xen_domid, 
PROT_READ|PROT_WRITE,
+                                       pfns, err, nb_pfn);
+#else
     vaddr_base = xc_map_foreign_bulk(xen_xc, xen_domid, PROT_READ|PROT_WRITE,
                                      pfns, err, nb_pfn);
+#endif
     if (vaddr_base == NULL) {
         perror("xc_map_foreign_bulk");
         exit(-1);


    Through I try to avoid modifing any code of linux-pv kernel, I
still have to commente out some code to make it run.

diff --git a/drivers/xen/xenfs/privcmd.c b/drivers/xen/xenfs/privcmd.c
index f80be7f..420f3b1 100644
--- a/drivers/xen/xenfs/privcmd.c
+++ b/drivers/xen/xenfs/privcmd.c
@@ -303,11 +309,10 @@ static long privcmd_ioctl_mmap_batch(void __user *udata)

        vma = find_vma(mm, m.addr);
        ret = -EINVAL;
+
        if (!vma ||
-           vma->vm_ops != &privcmd_vm_ops ||
            (m.addr != vma->vm_start) ||
-           ((m.addr + (nr_pages << PAGE_SHIFT)) != vma->vm_end) ||
-           !privcmd_enforce_singleshot_mapping(vma)) {
+           ((m.addr + (nr_pages << PAGE_SHIFT)) != vma->vm_end)) {
                up_write(&mm->mmap_sem);
                goto out;
        }


    The debug dump of qemu is:

(XEN) mm.c:945:d51 Error getting mfn 8ed68 (pfn 18000) from L1 entry
800000008ed68625 for l1e_owner=51, pg_owner=51
(XEN) mm.c:5044:d51 ptwr_emulate: fixing up invalid PAE PTE 800000008ed68625
(XEN) mm.c:945:d51 Error getting mfn 8ed68 (pfn 18000) from L1 entry
800000008ed68667 for l1e_owner=51, pg_owner=51
(XEN) mm.c:5049:d51 ptwr_emulate: could not get_page_from_l1e()
<1>BUG: unable to handle kernel <c>paging request<c> at c1f1b8f8
<1>IP: [<c011a7f0>] ptep_set_access_flags+0x40/0x80
*pdpt = 0000000001ee2001 <c>*pde = 0000000000008067 *pte = 8000000001f1b061
<0>Oops: 0003 [#1]
<0>last sysfs file: /sys/devices/virtual/vc/vcsa1/dev

Pid: 283, comm: qemu Not tainted (2.6.32.41 #6)
EIP: 0061:[<c011a7f0>] EFLAGS: 00010202 CPU: 0
EIP is at ptep_set_access_flags+0x40/0x80


     That seems the mapping is failed. (pfn 18000) is the first vram
page in hvm guest.


Jiageng Yu.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel