[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH v2] libs/gnttab: add FreeBSD handlers for the grant-table user-space device



This patch adds the headers and helpers for the FreeBSD gntdev, used in order
to map grants from remote domains and to allocate grants on behalf of the
current domain.

Current code has been tested with the QEMU/Qdisk backend.

Signed-off-by: Akshay Jaggi <akshay1994.leo@xxxxxxxxx>
[ added dummy stub for osdep_gnttab_grant_copy ]
Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
---
Cc: Wei Liu <wei.liu2@xxxxxxxxxx>
Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
---
Changes since v1:
 - Move some shared defines/macros to the private.h header.
 - Coding style fixes.
---
 tools/include/xen-sys/FreeBSD/gntdev.h | 191 ++++++++++++++++++++
 tools/libs/gnttab/Makefile             |   2 +-
 tools/libs/gnttab/freebsd.c            | 315 +++++++++++++++++++++++++++++++++
 tools/libs/gnttab/linux.c              |   9 -
 tools/libs/gnttab/private.h            |  10 ++
 5 files changed, 517 insertions(+), 10 deletions(-)
 create mode 100644 tools/include/xen-sys/FreeBSD/gntdev.h
 create mode 100644 tools/libs/gnttab/freebsd.c

diff --git a/tools/include/xen-sys/FreeBSD/gntdev.h 
b/tools/include/xen-sys/FreeBSD/gntdev.h
new file mode 100644
index 0000000..f3af9a4
--- /dev/null
+++ b/tools/include/xen-sys/FreeBSD/gntdev.h
@@ -0,0 +1,191 @@
+/*-
+ * Copyright (c) 2016 Akshay Jaggi <jaggi@xxxxxxxxxxx>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * gntdev.h
+ *
+ * Interface to /dev/xen/gntdev.
+ *
+ * This device provides the user with two kinds of functionalities:
+ * 1. Grant Allocation
+ *    Allocate a page of our own memory, and share it with a foreign domain.
+ * 2. Grant Mapping
+ *    Map a grant allocated by a foreign domain, into our own memory.
+ *
+ *
+ * Grant Allocation
+ *
+ * Steps to allocate a grant:
+ * 1. Do an `IOCTL_GNTDEV_ALLOC_GREF ioctl`, with
+ *     - `domid`, as the domain-id of the foreign domain
+ *     - `flags`, ORed with GNTDEV_ALLOC_FLAG_WRITABLE if you want the foreign
+ *       domain to have write access to the shared memory
+ *     - `count`, with the number of pages to share with the foreign domain
+ *
+ *    Ensure that the structure you allocate has enough memory to store
+ *    all the allocated grant-refs, i.e., you need to allocate
+ *    (sizeof(struct ioctl_gntdev_alloc_gref) + (count - 1)*sizeof(uint32_t))
+ *    bytes of memory.
+ *
+ * 2. Mmap the address given in `index` after a successful ioctl.
+ *    This will give you access to the granted pages.
+ *
+ * Note:
+ * 1. The grant is not removed until all three of the following conditions
+ *    are met
+ *     - The region is not mmaped. That is, munmap() has been called if
+ *       the region was mmapped previously.
+ *     - IOCTL_GNTDEV_DEALLOC_GREF ioctl has been performed. After you
+ *       perform this ioctl, you can no longer mmap or set notify on
+ *       the grant.
+ *     - The foreign domain has stopped using the grant.
+ * 2. Granted pages can only belong to one mmap region.
+ * 3. Every page of granted memory is a unit in itself. What this means
+ *    is that you can set a unmap notification for each of the granted
+ *    pages, individually; you can mmap and dealloc-ioctl a contiguous
+ *    range of allocated grants (even if alloc-ioctls were performed
+ *    individually), etc.
+ *
+ *
+ * Grant Mapping
+ *
+ * Steps to map a grant:
+ * 1. Do a `IOCTL_GNTDEV_MAP_GRANT_REF` ioctl, with
+ *     - `count`, as the number of foreign grants to map
+ *     - `refs[i].domid`, as the domain id of the foreign domain
+ *     - `refs[i].ref`, as the grant-ref for the grant to be mapped
+ *
+ * 2. Mmap the address given in `index` after a successful ioctl.
+ *    This will give you access to the mapped pages.
+ *
+ * Note:
+ * 1. The map hypercall is not made till the region is mmapped.
+ * 2. The unit is defined by the map ioctl. This means that only one
+ *    unmap notification can be set on a group of pages that were
+ *    mapped together in one ioctl, and also no single mmaping of contiguous
+ *    grant-maps is possible.
+ * 3. You can mmap the same grant-map region multiple times.
+ * 4. The grant is not unmapped until both of the following conditions are met
+ *     - The region is not mmaped. That is, munmap() has been called for
+ *       as many times as the grant was mmapped.
+ *     - IOCTL_GNTDEV_UNMAP_GRANT_REF ioctl has been called.
+ * 5. IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR ioctl gives index and count of
+ *    a grant-map from the virtual address of the location where the grant
+ *    is mmapped.
+ *
+ *
+ * IOCTL_GNTDEV_SET_UNMAP_NOTIFY
+ * This ioctl allows us to set notifications to be made when the grant is
+ * either unmapped (in case of a mapped grant), or when it is ready to be
+ * deallocated by us, ie, the grant is no more mmapped, and the dealloc
+ * ioctl has been called (in case of an allocated grant). OR `action` with
+ * the required notification masks, and fill in the appropriate fields.
+ *  - UNMAP_NOTIFY_CLEAR_BYTE clears the byte at `index`, where index is
+ *    the address of the byte in file address space.
+ *  - UNMAP_NOTIFY_SEND_EVENT sends an event channel notification on
+ *    `event_channel_port`
+ * In case of multiple notify ioctls, only the last one survives.
+ *
+ */
+
+#ifndef __XEN_GNTDEV_H__
+#define __XEN_GNTDEV_H__
+
+#include <sys/types.h>
+
+#define IOCTL_GNTDEV_SET_UNMAP_NOTIFY                                  \
+       _IOW('E', 0, struct ioctl_gntdev_unmap_notify)
+struct ioctl_gntdev_unmap_notify {
+    /* IN parameters */
+    uint64_t index;
+    uint32_t action;
+    uint32_t event_channel_port;
+};
+
+#define UNMAP_NOTIFY_CLEAR_BYTE 0x1
+#define UNMAP_NOTIFY_SEND_EVENT 0x2
+
+/*-------------------- Grant Allocation IOCTLs  
------------------------------*/
+
+#define IOCTL_GNTDEV_ALLOC_GREF                                                
\
+       _IOWR('E', 1, struct ioctl_gntdev_alloc_gref)
+struct ioctl_gntdev_alloc_gref {
+    /* IN parameters */
+    uint16_t domid;
+    uint16_t flags;
+    uint32_t count;
+    /* OUT parameters */
+    uint64_t index;
+    /* Variable OUT parameter */
+    uint32_t gref_ids[1];
+};
+
+#define GNTDEV_ALLOC_FLAG_WRITABLE 1
+
+#define IOCTL_GNTDEV_DEALLOC_GREF                                      \
+       _IOW('E', 2, struct ioctl_gntdev_dealloc_gref)
+struct ioctl_gntdev_dealloc_gref {
+    /* IN parameters */
+    uint64_t index;
+    uint32_t count;
+};
+
+/*-------------------- Grant Mapping IOCTLs  
---------------------------------*/
+
+struct ioctl_gntdev_grant_ref {
+    uint32_t domid;
+    uint32_t ref;
+};
+
+#define IOCTL_GNTDEV_MAP_GRANT_REF                                     \
+       _IOWR('E', 3, struct ioctl_gntdev_map_grant_ref)
+struct ioctl_gntdev_map_grant_ref {
+    /* IN parameters */
+    uint32_t count;
+    uint32_t pad0;
+    /* OUT parameters */
+    uint64_t index;
+    /* Variable IN parameter */
+    struct ioctl_gntdev_grant_ref refs[1];
+};
+
+#define IOCTL_GNTDEV_UNMAP_GRANT_REF                                   \
+       _IOW('E', 4, struct ioctl_gntdev_unmap_grant_ref)
+struct ioctl_gntdev_unmap_grant_ref {
+    /* IN parameters */
+    uint64_t index;
+    uint32_t count;
+};
+
+#define IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR                              \
+       _IOWR('E', 5, struct ioctl_gntdev_get_offset_for_vaddr)
+struct ioctl_gntdev_get_offset_for_vaddr {
+    /* IN parameters */
+    uint64_t vaddr;
+    /* OUT parameters */
+    uint64_t offset;
+    uint32_t count;
+};
+
+#endif /* __XEN_GNTDEV_H__ */
diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
index 95c2cd8..6a77f21 100644
--- a/tools/libs/gnttab/Makefile
+++ b/tools/libs/gnttab/Makefile
@@ -14,7 +14,7 @@ SRCS-GNTSHR            += gntshr_core.c
 
 SRCS-$(CONFIG_Linux)   += $(SRCS-GNTTAB) $(SRCS-GNTSHR) linux.c
 SRCS-$(CONFIG_MiniOS)  += $(SRCS-GNTTAB) gntshr_unimp.c minios.c
-SRCS-$(CONFIG_FreeBSD) += gnttab_unimp.c gntshr_unimp.c
+SRCS-$(CONFIG_FreeBSD) += $(SRCS-GNTTAB) $(SRCS-GNTSHR) freebsd.c
 SRCS-$(CONFIG_SunOS)   += gnttab_unimp.c gntshr_unimp.c
 SRCS-$(CONFIG_NetBSD)  += gnttab_unimp.c gntshr_unimp.c
 
diff --git a/tools/libs/gnttab/freebsd.c b/tools/libs/gnttab/freebsd.c
new file mode 100644
index 0000000..25c3dae
--- /dev/null
+++ b/tools/libs/gnttab/freebsd.c
@@ -0,0 +1,315 @@
+/*
+ * Copyright (c) 2007-2008, D G Murray <Derek.Murray@xxxxxxxxxxxx>
+ * Copyright (c) 2016-2017, Akshay Jaggi <jaggi@xxxxxxxxxxx>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Split out from linux.c
+ */
+
+#include <fcntl.h>
+#include <errno.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <xen/sys/gntdev.h>
+
+#include "private.h"
+
+#define DEVXEN "/dev/xen/gntdev"
+
+int osdep_gnttab_open(xengnttab_handle *xgt)
+{
+    int fd = open(DEVXEN, O_RDWR|O_CLOEXEC);
+
+    if ( fd == -1 )
+        return -1;
+    xgt->fd = fd;
+
+    return 0;
+}
+
+int osdep_gnttab_close(xengnttab_handle *xgt)
+{
+    if ( xgt->fd == -1 )
+        return 0;
+
+    return close(xgt->fd);
+}
+
+int osdep_gnttab_set_max_grants(xengnttab_handle *xgt, uint32_t count)
+{
+    return 0;
+}
+
+void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
+                             uint32_t count, int flags, int prot,
+                             uint32_t *domids, uint32_t *refs,
+                             uint32_t notify_offset,
+                             evtchn_port_t notify_port)
+{
+    uint32_t i;
+    int fd = xgt->fd;
+    struct ioctl_gntdev_map_grant_ref *map;
+    void *addr = NULL;
+    int domids_stride;
+    unsigned int map_size = ROUNDUP((sizeof(*map) + (count - 1) *
+                                    sizeof(struct ioctl_gntdev_map_grant_ref)),
+                                    PAGE_SHIFT);
+
+    domids_stride = (flags & XENGNTTAB_GRANT_MAP_SINGLE_DOMAIN) ? 0 : 1;
+    if ( map_size <= PAGE_SIZE )
+        map = malloc(sizeof(*map) +
+                     (count - 1) * sizeof(struct ioctl_gntdev_map_grant_ref));
+    else
+    {
+        map = mmap(NULL, map_size, PROT_READ | PROT_WRITE,
+                   MAP_PRIVATE | MAP_ANON, -1, 0);
+        if ( map == MAP_FAILED )
+        {
+            GTERROR(xgt->logger, "anon mmap of map failed");
+            return NULL;
+        }
+    }
+
+    for ( i = 0; i < count; i++ )
+    {
+        map->refs[i].domid = domids[i * domids_stride];
+        map->refs[i].ref = refs[i];
+    }
+
+    map->count = count;
+
+    if ( ioctl(fd, IOCTL_GNTDEV_MAP_GRANT_REF, map) )
+    {
+        GTERROR(xgt->logger, "ioctl MAP_GRANT_REF failed");
+        goto out;
+    }
+
+    addr = mmap(NULL, PAGE_SIZE * count, prot, MAP_SHARED, fd,
+                map->index);
+    if ( addr != MAP_FAILED )
+    {
+        int rv = 0;
+        struct ioctl_gntdev_unmap_notify notify;
+
+        notify.index = map->index;
+        notify.action = 0;
+        if ( notify_offset < PAGE_SIZE * count )
+        {
+            notify.index += notify_offset;
+            notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
+        }
+        if ( notify_port != -1 )
+        {
+            notify.event_channel_port = notify_port;
+            notify.action |= UNMAP_NOTIFY_SEND_EVENT;
+        }
+        if ( notify.action )
+            rv = ioctl(fd, IOCTL_GNTDEV_SET_UNMAP_NOTIFY, &notify);
+        if ( rv )
+        {
+            GTERROR(xgt->logger, "ioctl SET_UNMAP_NOTIFY failed");
+            munmap(addr, count * PAGE_SIZE);
+            addr = MAP_FAILED;
+        }
+    }
+    if ( addr == MAP_FAILED )
+    {
+        int saved_errno = errno;
+        struct ioctl_gntdev_unmap_grant_ref unmap_grant;
+
+        /* Unmap the driver slots used to store the grant information. */
+        GTERROR(xgt->logger, "mmap failed");
+        unmap_grant.index = map->index;
+        unmap_grant.count = count;
+        ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &unmap_grant);
+        errno = saved_errno;
+        addr = NULL;
+    }
+
+ out:
+    if ( map_size > PAGE_SIZE )
+        munmap(map, map_size);
+    else
+        free(map);
+
+    return addr;
+}
+
+int osdep_gnttab_unmap(xengnttab_handle *xgt,
+                       void *start_address,
+                       uint32_t count)
+{
+    int rc;
+    int fd = xgt->fd;
+    struct ioctl_gntdev_unmap_grant_ref unmap_grant;
+    struct ioctl_gntdev_get_offset_for_vaddr get_offset;
+
+    if ( start_address == NULL )
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    /*
+     * First, it is necessary to get the offset which was initially used to
+     * mmap() the pages.
+     */
+    get_offset.vaddr = (unsigned long)start_address;
+    if ( (rc = ioctl(fd, IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR,
+                     &get_offset)) )
+        return rc;
+
+    if ( get_offset.count != count )
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    /* Next, unmap the memory. */
+    if ( (rc = munmap(start_address, count * PAGE_SIZE)) )
+        return rc;
+
+    /* Finally, unmap the driver slots used to store the grant information. */
+    unmap_grant.index = get_offset.offset;
+    unmap_grant.count = count;
+    if ( (rc = ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &unmap_grant)) )
+        return rc;
+
+    return 0;
+}
+
+int osdep_gnttab_grant_copy(xengnttab_handle *xgt,
+                            uint32_t count,
+                            xengnttab_grant_copy_segment_t *segs)
+{
+    errno = ENOSYS;
+    return -1;
+}
+
+int osdep_gntshr_open(xengntshr_handle *xgs)
+{
+
+    int fd = open(DEVXEN, O_RDWR);
+
+    if ( fd == -1 )
+        return -1;
+    xgs->fd = fd;
+
+    return 0;
+}
+
+int osdep_gntshr_close(xengntshr_handle *xgs)
+{
+    if ( xgs->fd == -1 )
+        return 0;
+
+    return close(xgs->fd);
+}
+
+void *osdep_gntshr_share_pages(xengntshr_handle *xgs,
+                               uint32_t domid, int count,
+                               uint32_t *refs, int writable,
+                               uint32_t notify_offset,
+                               evtchn_port_t notify_port)
+{
+    int err;
+    int fd = xgs->fd;
+    void *area = NULL;
+    struct ioctl_gntdev_unmap_notify notify;
+    struct ioctl_gntdev_dealloc_gref gref_drop;
+    struct ioctl_gntdev_alloc_gref *gref_info = NULL;
+
+    gref_info = malloc(sizeof(*gref_info) + count * sizeof(uint32_t));
+    if ( gref_info == NULL )
+        return NULL;
+    gref_info->domid = domid;
+    gref_info->flags = writable ? GNTDEV_ALLOC_FLAG_WRITABLE : 0;
+    gref_info->count = count;
+
+    err = ioctl(fd, IOCTL_GNTDEV_ALLOC_GREF, gref_info);
+    if ( err )
+    {
+        GSERROR(xgs->logger, "ioctl failed");
+        goto out;
+    }
+
+    area = mmap(NULL, count * PAGE_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
+                fd, gref_info->index);
+
+    if ( area == MAP_FAILED )
+    {
+        area = NULL;
+        GSERROR(xgs->logger, "mmap failed");
+        goto out_remove_fdmap;
+    }
+
+    notify.index = gref_info->index;
+    notify.action = 0;
+    if ( notify_offset < PAGE_SIZE * count )
+    {
+        notify.index += notify_offset;
+        notify.action |= UNMAP_NOTIFY_CLEAR_BYTE;
+    }
+    if ( notify_port != -1 )
+    {
+        notify.event_channel_port = notify_port;
+        notify.action |= UNMAP_NOTIFY_SEND_EVENT;
+    }
+    if ( notify.action )
+        err = ioctl(fd, IOCTL_GNTDEV_SET_UNMAP_NOTIFY, &notify);
+    if ( err )
+    {
+        GSERROR(xgs->logger, "ioctl SET_UNMAP_NOTIFY failed");
+        munmap(area, count * PAGE_SIZE);
+        area = NULL;
+    }
+
+    memcpy(refs, gref_info->gref_ids, count * sizeof(uint32_t));
+
+ out_remove_fdmap:
+    /*
+     * Removing the mapping from the file descriptor does not cause the
+     * pages to be deallocated until the mapping is removed.
+     */
+    gref_drop.index = gref_info->index;
+    gref_drop.count = count;
+    ioctl(fd, IOCTL_GNTDEV_DEALLOC_GREF, &gref_drop);
+ out:
+    free(gref_info);
+
+    return area;
+}
+
+int osdep_gntshr_unshare(xengntshr_handle *xgs,
+                         void *start_address, uint32_t count)
+{
+    return munmap(start_address, count * PAGE_SIZE);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libs/gnttab/linux.c b/tools/libs/gnttab/linux.c
index 69b7e26..9d5dc74 100644
--- a/tools/libs/gnttab/linux.c
+++ b/tools/libs/gnttab/linux.c
@@ -37,15 +37,6 @@
 
 #define DEVXEN "/dev/xen/"
 
-#define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
-
-#define GTERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gnttab", _f)
-#define GSERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gntshr", _f)
-
-#define PAGE_SHIFT           12
-#define PAGE_SIZE            (1UL << PAGE_SHIFT)
-#define PAGE_MASK            (~(PAGE_SIZE-1))
-
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 #endif
diff --git a/tools/libs/gnttab/private.h b/tools/libs/gnttab/private.h
index d6c5594..1416194 100644
--- a/tools/libs/gnttab/private.h
+++ b/tools/libs/gnttab/private.h
@@ -4,6 +4,16 @@
 #include <xentoollog.h>
 #include <xengnttab.h>
 
+/* Set of macros/defines used by both Linux and FreeBSD */
+#define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
+
+#define GTERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gnttab", _f)
+#define GSERROR(_l, _f...) xtl_log(_l, XTL_ERROR, errno, "gntshr", _f)
+
+#define PAGE_SHIFT           12
+#define PAGE_SIZE            (1UL << PAGE_SHIFT)
+#define PAGE_MASK            (~(PAGE_SIZE-1))
+
 struct xengntdev_handle {
     xentoollog_logger *logger, *logger_tofree;
     int fd;
-- 
2.10.1 (Apple Git-78)


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.