# HG changeset patch
# User kaf24@xxxxxxxxxxxxxxxxxxxx
# Node ID 750ad97f37b0a49451c9b887c8ccb9134cc8a1ec
# Parent c0796e18b6a45f0352770e700e3f6cae028bd2e3
Split up docs. Signed-off-by: Robb Romans <3r@xxxxxxxxxx>
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/Makefile
--- a/docs/Makefile Tue Sep 20 09:08:26 2005
+++ b/docs/Makefile Tue Sep 20 09:17:33 2005
@@ -12,7 +12,7 @@
pkgdocdir := /usr/share/doc/xen
-DOC_TEX := $(wildcard src/*.tex)
+DOC_TEX := src/user.tex src/interface.tex
DOC_PS := $(patsubst src/%.tex,ps/%.ps,$(DOC_TEX))
DOC_PDF := $(patsubst src/%.tex,pdf/%.pdf,$(DOC_TEX))
DOC_HTML := $(patsubst src/%.tex,html/%/index.html,$(DOC_TEX))
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface.tex
--- a/docs/src/interface.tex Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface.tex Tue Sep 20 09:17:33 2005
@@ -87,1084 +87,23 @@
mechanism and policy within the system.
+%% chapter Virtual Architecture moved to architecture.tex
+\include{src/interface/architecture}
-\chapter{Virtual Architecture}
+%% chapter Memory moved to memory.tex
+\include{src/interface/memory}
-On a Xen-based system, the hypervisor itself runs in {\it ring 0}. It
-has full access to the physical memory available in the system and is
-responsible for allocating portions of it to the domains. Guest
-operating systems run in and use {\it rings 1}, {\it 2} and {\it 3} as
-they see fit. Segmentation is used to prevent the guest OS from
-accessing the portion of the address space that is reserved for
-Xen. We expect most guest operating systems will use ring 1 for their
-own operation and place applications in ring 3.
+%% chapter Devices moved to devices.tex
+\include{src/interface/devices}
-In this chapter we consider the basic virtual architecture provided
-by Xen: the basic CPU state, exception and interrupt handling, and
-time. Other aspects such as memory and device access are discussed
-in later chapters.
-
-\section{CPU state}
-
-All privileged state must be handled by Xen. The guest OS has no
-direct access to CR3 and is not permitted to update privileged bits in
-EFLAGS. Guest OSes use \emph{hypercalls} to invoke operations in Xen;
-these are analogous to system calls but occur from ring 1 to ring 0.
-
-A list of all hypercalls is given in Appendix~\ref{a:hypercalls}.
-
-
-
-\section{Exceptions}
-
-A virtual IDT is provided --- a domain can submit a table of trap
-handlers to Xen via the {\tt set\_trap\_table()} hypercall. Most trap
-handlers are identical to native x86 handlers, although the page-fault
-handler is somewhat different.
-
-
-\section{Interrupts and events}
-
-Interrupts are virtualized by mapping them to \emph{events}, which are
-delivered asynchronously to the target domain using a callback
-supplied via the {\tt set\_callbacks()} hypercall. A guest OS can map
-these events onto its standard interrupt dispatch mechanisms. Xen is
-responsible for determining the target domain that will handle each
-physical interrupt source. For more details on the binding of event
-sources to events, see Chapter~\ref{c:devices}.
-
-
-
-\section{Time}
-
-Guest operating systems need to be aware of the passage of both real
-(or wallclock) time and their own `virtual time' (the time for
-which they have been executing). Furthermore, Xen has a notion of
-time which is used for scheduling. The following notions of
-time are provided:
-
-\begin{description}
-\item[Cycle counter time.]
-
-This provides a fine-grained time reference. The cycle counter time is
-used to accurately extrapolate the other time references. On SMP machines
-it is currently assumed that the cycle counter time is synchronized between
-CPUs. The current x86-based implementation achieves this within inter-CPU
-communication latencies.
-
-\item[System time.]
-
-This is a 64-bit counter which holds the number of nanoseconds that
-have elapsed since system boot.
-
-
-\item[Wall clock time.]
-
-This is the time of day in a Unix-style {\tt struct timeval} (seconds
-and microseconds since 1 January 1970, adjusted by leap seconds). An
-NTP client hosted by {\it domain 0} can keep this value accurate.
-
-
-\item[Domain virtual time.]
-
-This progresses at the same pace as system time, but only while a
-domain is executing --- it stops while a domain is de-scheduled.
-Therefore the share of the CPU that a domain receives is indicated by
-the rate at which its virtual time increases.
-
-\end{description}
-
-
-Xen exports timestamps for system time and wall-clock time to guest
-operating systems through a shared page of memory. Xen also provides
-the cycle counter time at the instant the timestamps were calculated,
-and the CPU frequency in Hertz. This allows the guest to extrapolate
-system and wall-clock times accurately based on the current cycle
-counter time.
-
-Since all time stamps need to be updated and read \emph{atomically}
-two version numbers are also stored in the shared info page. The
-first is incremented prior to an update, while the second is only
-incremented afterwards. Thus a guest can be sure that it read a consistent
-state by checking the two version numbers are equal.
-
-Xen includes a periodic ticker which sends a timer event to the
-currently executing domain every 10ms. The Xen scheduler also sends a
-timer event whenever a domain is scheduled; this allows the guest OS
-to adjust for the time that has passed while it has been inactive. In
-addition, Xen allows each domain to request that they receive a timer
-event sent at a specified system time by using the {\tt
-set\_timer\_op()} hypercall. Guest OSes may use this timer to
-implement timeout values when they block.
-
-
-
-%% % akw: demoting this to a section -- not sure if there is any point
-%% % though, maybe just remove it.
-
-\section{Xen CPU Scheduling}
-
-Xen offers a uniform API for CPU schedulers. It is possible to choose
-from a number of schedulers at boot and it should be easy to add more.
-The BVT, Atropos and Round Robin schedulers are part of the normal
-Xen distribution. BVT provides proportional fair shares of the CPU to
-the running domains. Atropos can be used to reserve absolute shares
-of the CPU for each domain. Round-robin is provided as an example of
-Xen's internal scheduler API.
-
-\paragraph*{Note: SMP host support}
-Xen has always supported SMP host systems. Domains are statically assigned to
-CPUs, either at creation time or when manually pinning to a particular CPU.
-The current schedulers then run locally on each CPU to decide which of the
-assigned domains should be run there. The user-level control software
-can be used to perform coarse-grain load-balancing between CPUs.
-
-
-%% More information on the characteristics and use of these schedulers is
-%% available in {\tt Sched-HOWTO.txt}.
-
-
-\section{Privileged operations}
-
-Xen exports an extended interface to privileged domains (viz.\ {\it
- Domain 0}). This allows such domains to build and boot other domains
-on the server, and provides control interfaces for managing
-scheduling, memory, networking, and block devices.
-
-
-\chapter{Memory}
-\label{c:memory}
-
-Xen is responsible for managing the allocation of physical memory to
-domains, and for ensuring safe use of the paging and segmentation
-hardware.
-
-
-\section{Memory Allocation}
-
-
-Xen resides within a small fixed portion of physical memory; it also
-reserves the top 64MB of every virtual address space. The remaining
-physical memory is available for allocation to domains at a page
-granularity. Xen tracks the ownership and use of each page, which
-allows it to enforce secure partitioning between domains.
-
-Each domain has a maximum and current physical memory allocation.
-A guest OS may run a `balloon driver' to dynamically adjust its
-current memory allocation up to its limit.
-
-
-%% XXX SMH: I use machine and physical in the next section (which
-%% is kinda required for consistency with code); wonder if this
-%% section should use same terms?
-%%
-%% Probably.
-%%
-%% Merging this and below section at some point prob makes sense.
-
-\section{Pseudo-Physical Memory}
-
-Since physical memory is allocated and freed on a page granularity,
-there is no guarantee that a domain will receive a contiguous stretch
-of physical memory. However most operating systems do not have good
-support for operating in a fragmented physical address space. To aid
-porting such operating systems to run on top of Xen, we make a
-distinction between \emph{machine memory} and \emph{pseudo-physical
-memory}.
-
-Put simply, machine memory refers to the entire amount of memory
-installed in the machine, including that reserved by Xen, in use by
-various domains, or currently unallocated. We consider machine memory
-to comprise a set of 4K \emph{machine page frames} numbered
-consecutively starting from 0. Machine frame numbers mean the same
-within Xen or any domain.
-
-Pseudo-physical memory, on the other hand, is a per-domain
-abstraction. It allows a guest operating system to consider its memory
-allocation to consist of a contiguous range of physical page frames
-starting at physical frame 0, despite the fact that the underlying
-machine page frames may be sparsely allocated and in any order.
-
-To achieve this, Xen maintains a globally readable {\it
-machine-to-physical} table which records the mapping from machine page
-frames to pseudo-physical ones. In addition, each domain is supplied
-with a {\it physical-to-machine} table which performs the inverse
-mapping. Clearly the machine-to-physical table has size proportional
-to the amount of RAM installed in the machine, while each
-physical-to-machine table has size proportional to the memory
-allocation of the given domain.
-
-Architecture dependent code in guest operating systems can then use
-the two tables to provide the abstraction of pseudo-physical
-memory. In general, only certain specialized parts of the operating
-system (such as page table management) needs to understand the
-difference between machine and pseudo-physical addresses.
-
-\section{Page Table Updates}
-
-In the default mode of operation, Xen enforces read-only access to
-page tables and requires guest operating systems to explicitly request
-any modifications. Xen validates all such requests and only applies
-updates that it deems safe. This is necessary to prevent domains from
-adding arbitrary mappings to their page tables.
-
-To aid validation, Xen associates a type and reference count with each
-memory page. A page has one of the following
-mutually-exclusive types at any point in time: page directory ({\sf
-PD}), page table ({\sf PT}), local descriptor table ({\sf LDT}),
-global descriptor table ({\sf GDT}), or writable ({\sf RW}). Note that
-a guest OS may always create readable mappings of its own memory
-regardless of its current type.
-%%% XXX: possibly explain more about ref count 'lifecyle' here?
-This mechanism is used to
-maintain the invariants required for safety; for example, a domain
-cannot have a writable mapping to any part of a page table as this
-would require the page concerned to simultaneously be of types {\sf
- PT} and {\sf RW}.
-
-
-%\section{Writable Page Tables}
-
-Xen also provides an alternative mode of operation in which guests be
-have the illusion that their page tables are directly writable. Of
-course this is not really the case, since Xen must still validate
-modifications to ensure secure partitioning. To this end, Xen traps
-any write attempt to a memory page of type {\sf PT} (i.e., that is
-currently part of a page table). If such an access occurs, Xen
-temporarily allows write access to that page while at the same time
-{\em disconnecting} it from the page table that is currently in
-use. This allows the guest to safely make updates to the page because
-the newly-updated entries cannot be used by the MMU until Xen
-revalidates and reconnects the page.
-Reconnection occurs automatically in a number of situations: for
-example, when the guest modifies a different page-table page, when the
-domain is preempted, or whenever the guest uses Xen's explicit
-page-table update interfaces.
-
-Finally, Xen also supports a form of \emph{shadow page tables} in
-which the guest OS uses a independent copy of page tables which are
-unknown to the hardware (i.e.\ which are never pointed to by {\tt
-cr3}). Instead Xen propagates changes made to the guest's tables to the
-real ones, and vice versa. This is useful for logging page writes
-(e.g.\ for live migration or checkpoint). A full version of the shadow
-page tables also allows guest OS porting with less effort.
-
-\section{Segment Descriptor Tables}
-
-On boot a guest is supplied with a default GDT, which does not reside
-within its own memory allocation. If the guest wishes to use other
-than the default `flat' ring-1 and ring-3 segments that this GDT
-provides, it must register a custom GDT and/or LDT with Xen,
-allocated from its own memory. Note that a number of GDT
-entries are reserved by Xen -- any custom GDT must also include
-sufficient space for these entries.
-
-For example, the following hypercall is used to specify a new GDT:
-
-\begin{quote}
-int {\bf set\_gdt}(unsigned long *{\em frame\_list}, int {\em entries})
-
-{\em frame\_list}: An array of up to 16 machine page frames within
-which the GDT resides. Any frame registered as a GDT frame may only
-be mapped read-only within the guest's address space (e.g., no
-writable mappings, no use as a page-table page, and so on).
-
-{\em entries}: The number of descriptor-entry slots in the GDT. Note
-that the table must be large enough to contain Xen's reserved entries;
-thus we must have `{\em entries $>$ LAST\_RESERVED\_GDT\_ENTRY}\ '.
-Note also that, after registering the GDT, slots {\em FIRST\_} through
-{\em LAST\_RESERVED\_GDT\_ENTRY} are no longer usable by the guest and
-may be overwritten by Xen.
-\end{quote}
-
-The LDT is updated via the generic MMU update mechanism (i.e., via
-the {\tt mmu\_update()} hypercall.
-
-\section{Start of Day}
-
-The start-of-day environment for guest operating systems is rather
-different to that provided by the underlying hardware. In particular,
-the processor is already executing in protected mode with paging
-enabled.
-
-{\it Domain 0} is created and booted by Xen itself. For all subsequent
-domains, the analogue of the boot-loader is the {\it domain builder},
-user-space software running in {\it domain 0}. The domain builder
-is responsible for building the initial page tables for a domain
-and loading its kernel image at the appropriate virtual address.
-
-
-
-\chapter{Devices}
-\label{c:devices}
-
-Devices such as network and disk are exported to guests using a
-split device driver. The device driver domain, which accesses the
-physical device directly also runs a {\em backend} driver, serving
-requests to that device from guests. Each guest will use a simple
-{\em frontend} driver, to access the backend. Communication between these
-domains is composed of two parts: First, data is placed onto a shared
-memory page between the domains. Second, an event channel between the
-two domains is used to pass notification that data is outstanding.
-This separation of notification from data transfer allows message
-batching, and results in very efficient device access.
-
-Event channels are used extensively in device virtualization; each
-domain has a number of end-points or \emph{ports} each of which
-may be bound to one of the following \emph{event sources}:
-\begin{itemize}
- \item a physical interrupt from a real device,
- \item a virtual interrupt (callback) from Xen, or
- \item a signal from another domain
-\end{itemize}
-
-Events are lightweight and do not carry much information beyond
-the source of the notification. Hence when performing bulk data
-transfer, events are typically used as synchronization primitives
-over a shared memory transport. Event channels are managed via
-the {\tt event\_channel\_op()} hypercall; for more details see
-Section~\ref{s:idc}.
-
-This chapter focuses on some individual device interfaces
-available to Xen guests.
-
-\section{Network I/O}
-
-Virtual network device services are provided by shared memory
-communication with a backend domain. From the point of view of
-other domains, the backend may be viewed as a virtual ethernet switch
-element with each domain having one or more virtual network interfaces
-connected to it.
-
-\subsection{Backend Packet Handling}
-
-The backend driver is responsible for a variety of actions relating to
-the transmission and reception of packets from the physical device.
-With regard to transmission, the backend performs these key actions:
-
-\begin{itemize}
-\item {\bf Validation:} To ensure that domains do not attempt to
- generate invalid (e.g. spoofed) traffic, the backend driver may
- validate headers ensuring that source MAC and IP addresses match the
- interface that they have been sent from.
-
- Validation functions can be configured using standard firewall rules
- ({\small{\tt iptables}} in the case of Linux).
-
-\item {\bf Scheduling:} Since a number of domains can share a single
- physical network interface, the backend must mediate access when
- several domains each have packets queued for transmission. This
- general scheduling function subsumes basic shaping or rate-limiting
- schemes.
-
-\item {\bf Logging and Accounting:} The backend domain can be
- configured with classifier rules that control how packets are
- accounted or logged. For example, log messages might be generated
- whenever a domain attempts to send a TCP packet containing a SYN.
-\end{itemize}
-
-On receipt of incoming packets, the backend acts as a simple
-demultiplexer: Packets are passed to the appropriate virtual
-interface after any necessary logging and accounting have been carried
-out.
-
-\subsection{Data Transfer}
-
-Each virtual interface uses two ``descriptor rings'', one for transmit,
-the other for receive. Each descriptor identifies a block of contiguous
-physical memory allocated to the domain.
-
-The transmit ring carries packets to transmit from the guest to the
-backend domain. The return path of the transmit ring carries messages
-indicating that the contents have been physically transmitted and the
-backend no longer requires the associated pages of memory.
-
-To receive packets, the guest places descriptors of unused pages on
-the receive ring. The backend will return received packets by
-exchanging these pages in the domain's memory with new pages
-containing the received data, and passing back descriptors regarding
-the new packets on the ring. This zero-copy approach allows the
-backend to maintain a pool of free pages to receive packets into, and
-then deliver them to appropriate domains after examining their
-headers.
-
-%
-%Real physical addresses are used throughout, with the domain performing
-%translation from pseudo-physical addresses if that is necessary.
-
-If a domain does not keep its receive ring stocked with empty buffers then
-packets destined to it may be dropped. This provides some defence against
-receive livelock problems because an overload domain will cease to receive
-further data. Similarly, on the transmit path, it provides the application
-with feedback on the rate at which packets are able to leave the system.
-
-
-Flow control on rings is achieved by including a pair of producer
-indexes on the shared ring page. Each side will maintain a private
-consumer index indicating the next outstanding message. In this
-manner, the domains cooperate to divide the ring into two message
-lists, one in each direction. Notification is decoupled from the
-immediate placement of new messages on the ring; the event channel
-will be used to generate notification when {\em either} a certain
-number of outstanding messages are queued, {\em or} a specified number
-of nanoseconds have elapsed since the oldest message was placed on the
-ring.
-
-% Not sure if my version is any better -- here is what was here before:
-%% Synchronization between the backend domain and the guest is achieved using
-%% counters held in shared memory that is accessible to both. Each ring has
-%% associated producer and consumer indices indicating the area in the ring
-%% that holds descriptors that contain data. After receiving {\it n} packets
-%% or {\t nanoseconds} after receiving the first packet, the hypervisor sends
-%% an event to the domain.
-
-\section{Block I/O}
-
-All guest OS disk access goes through the virtual block device VBD
-interface. This interface allows domains access to portions of block
-storage devices visible to the the block backend device. The VBD
-interface is a split driver, similar to the network interface
-described above. A single shared memory ring is used between the
-frontend and backend drivers, across which read and write messages are
-sent.
-
-Any block device accessible to the backend domain, including
-network-based block (iSCSI, *NBD, etc), loopback and LVM/MD devices,
-can be exported as a VBD. Each VBD is mapped to a device node in the
-guest, specified in the guest's startup configuration.
-
-Old (Xen 1.2) virtual disks are not supported under Xen 2.0, since
-similar functionality can be achieved using the more complete LVM
-system, which is already in widespread use.
-
-\subsection{Data Transfer}
-
-The single ring between the guest and the block backend supports three
-messages:
-
-\begin{description}
-\item [{\small {\tt PROBE}}:] Return a list of the VBDs available to this guest
- from the backend. The request includes a descriptor of a free page
- into which the reply will be written by the backend.
-
-\item [{\small {\tt READ}}:] Read data from the specified block device. The
- front end identifies the device and location to read from and
- attaches pages for the data to be copied to (typically via DMA from
- the device). The backend acknowledges completed read requests as
- they finish.
-
-\item [{\small {\tt WRITE}}:] Write data to the specified block device. This
- functions essentially as {\small {\tt READ}}, except that the data moves to
- the device instead of from it.
-\end{description}
-
-% um... some old text
-%% In overview, the same style of descriptor-ring that is used for
-%% network packets is used here. Each domain has one ring that carries
-%% operation requests to the hypervisor and carries the results back
-%% again.
-
-%% Rather than copying data, the backend simply maps the domain's buffers
-%% in order to enable direct DMA to them. The act of mapping the buffers
-%% also increases the reference counts of the underlying pages, so that
-%% the unprivileged domain cannot try to return them to the hypervisor,
-%% install them as page tables, or any other unsafe behaviour.
-%% %block API here
-
-
-\chapter{Further Information}
-
-
-If you have questions that are not answered by this manual, the
-sources of information listed below may be of interest to you. Note
-that bug reports, suggestions and contributions related to the
-software (or the documentation) should be sent to the Xen developers'
-mailing list (address below).
-
-\section{Other documentation}
-
-If you are mainly interested in using (rather than developing for)
-Xen, the {\em Xen Users' Manual} is distributed in the {\tt docs/}
-directory of the Xen source distribution.
-
-% Various HOWTOs are also available in {\tt docs/HOWTOS}.
-
-\section{Online references}
-
-The official Xen web site is found at:
-\begin{quote}
-{\tt http://www.cl.cam.ac.uk/Research/SRG/netos/xen/}
-\end{quote}
-
-This contains links to the latest versions of all on-line
-documentation.
-
-\section{Mailing lists}
-
-There are currently four official Xen mailing lists:
-
-\begin{description}
-\item[xen-devel@xxxxxxxxxxxxxxxxxxx] Used for development
-discussions and bug reports. Subscribe at: \\
-{\small {\tt http://lists.xensource.com/xen-devel}}
-\item[xen-users@xxxxxxxxxxxxxxxxxxx] Used for installation and usage
-discussions and requests for help. Subscribe at: \\
-{\small {\tt http://lists.xensource.com/xen-users}}
-\item[xen-announce@xxxxxxxxxxxxxxxxxxx] Used for announcements only.
-Subscribe at: \\
-{\small {\tt http://lists.xensource.com/xen-announce}}
-\item[xen-changelog@xxxxxxxxxxxxxxxxxxx] Changelog feed
-from the unstable and 2.0 trees - developer oriented. Subscribe at: \\
-{\small {\tt http://lists.xensource.com/xen-changelog}}
-\end{description}
-
-Of these, xen-devel is the most active.
-
-
+%% chapter Further Information moved to further_info.tex
+\include{src/interface/further_info}
\appendix
-%\newcommand{\hypercall}[1]{\vspace{5mm}{\large\sf #1}}
-
-
-
-
-
-\newcommand{\hypercall}[1]{\vspace{2mm}{\sf #1}}
-
-
-
-
-
-
-\chapter{Xen Hypercalls}
-\label{a:hypercalls}
-
-Hypercalls represent the procedural interface to Xen; this appendix
-categorizes and describes the current set of hypercalls.
-
-\section{Invoking Hypercalls}
-
-Hypercalls are invoked in a manner analogous to system calls in a
-conventional operating system; a software interrupt is issued which
-vectors to an entry point within Xen. On x86\_32 machines the
-instruction required is {\tt int \$82}; the (real) IDT is setup so
-that this may only be issued from within ring 1. The particular
-hypercall to be invoked is contained in {\tt EAX} --- a list
-mapping these values to symbolic hypercall names can be found
-in {\tt xen/include/public/xen.h}.
-
-On some occasions a set of hypercalls will be required to carry
-out a higher-level function; a good example is when a guest
-operating wishes to context switch to a new process which
-requires updating various privileged CPU state. As an optimization
-for these cases, there is a generic mechanism to issue a set of
-hypercalls as a batch:
-
-\begin{quote}
-\hypercall{multicall(void *call\_list, int nr\_calls)}
-
-Execute a series of hypervisor calls; {\tt nr\_calls} is the length of
-the array of {\tt multicall\_entry\_t} structures pointed to be {\tt
-call\_list}. Each entry contains the hypercall operation code followed
-by up to 7 word-sized arguments.
-\end{quote}
-
-Note that multicalls are provided purely as an optimization; there is
-no requirement to use them when first porting a guest operating
-system.
-
-
-\section{Virtual CPU Setup}
-
-At start of day, a guest operating system needs to setup the virtual
-CPU it is executing on. This includes installing vectors for the
-virtual IDT so that the guest OS can handle interrupts, page faults,
-etc. However the very first thing a guest OS must setup is a pair
-of hypervisor callbacks: these are the entry points which Xen will
-use when it wishes to notify the guest OS of an occurrence.
-
-\begin{quote}
-\hypercall{set\_callbacks(unsigned long event\_selector, unsigned long
- event\_address, unsigned long failsafe\_selector, unsigned long
- failsafe\_address) }
-
-Register the normal (``event'') and failsafe callbacks for
-event processing. In each case the code segment selector and
-address within that segment are provided. The selectors must
-have RPL 1; in XenLinux we simply use the kernel's CS for both
-{\tt event\_selector} and {\tt failsafe\_selector}.
-
-The value {\tt event\_address} specifies the address of the guest OSes
-event handling and dispatch routine; the {\tt failsafe\_address}
-specifies a separate entry point which is used only if a fault occurs
-when Xen attempts to use the normal callback.
-\end{quote}
-
-
-After installing the hypervisor callbacks, the guest OS can
-install a `virtual IDT' by using the following hypercall:
-
-\begin{quote}
-\hypercall{set\_trap\_table(trap\_info\_t *table)}
-
-Install one or more entries into the per-domain
-trap handler table (essentially a software version of the IDT).
-Each entry in the array pointed to by {\tt table} includes the
-exception vector number with the corresponding segment selector
-and entry point. Most guest OSes can use the same handlers on
-Xen as when running on the real hardware; an exception is the
-page fault handler (exception vector 14) where a modified
-stack-frame layout is used.
-
-
-\end{quote}
-
-
-
-\section{Scheduling and Timer}
-
-Domains are preemptively scheduled by Xen according to the
-parameters installed by domain 0 (see Section~\ref{s:dom0ops}).
-In addition, however, a domain may choose to explicitly
-control certain behavior with the following hypercall:
-
-\begin{quote}
-\hypercall{sched\_op(unsigned long op)}
-
-Request scheduling operation from hypervisor. The options are: {\it
-yield}, {\it block}, and {\it shutdown}. {\it yield} keeps the
-calling domain runnable but may cause a reschedule if other domains
-are runnable. {\it block} removes the calling domain from the run
-queue and cause is to sleeps until an event is delivered to it. {\it
-shutdown} is used to end the domain's execution; the caller can
-additionally specify whether the domain should reboot, halt or
-suspend.
-\end{quote}
-
-To aid the implementation of a process scheduler within a guest OS,
-Xen provides a virtual programmable timer:
-
-\begin{quote}
-\hypercall{set\_timer\_op(uint64\_t timeout)}
-
-Request a timer event to be sent at the specified system time (time
-in nanoseconds since system boot). The hypercall actually passes the
-64-bit timeout value as a pair of 32-bit values.
-
-\end{quote}
-
-Note that calling {\tt set\_timer\_op()} prior to {\tt sched\_op}
-allows block-with-timeout semantics.
-
-
-\section{Page Table Management}
-
-Since guest operating systems have read-only access to their page
-tables, Xen must be involved when making any changes. The following
-multi-purpose hypercall can be used to modify page-table entries,
-update the machine-to-physical mapping table, flush the TLB, install
-a new page-table base pointer, and more.
-
-\begin{quote}
-\hypercall{mmu\_update(mmu\_update\_t *req, int count, int *success\_count)}
-
-Update the page table for the domain; a set of {\tt count} updates are
-submitted for processing in a batch, with {\tt success\_count} being
-updated to report the number of successful updates.
-
-Each element of {\tt req[]} contains a pointer (address) and value;
-the least significant 2-bits of the pointer are used to distinguish
-the type of update requested as follows:
-\begin{description}
-
-\item[\it MMU\_NORMAL\_PT\_UPDATE:] update a page directory entry or
-page table entry to the associated value; Xen will check that the
-update is safe, as described in Chapter~\ref{c:memory}.
-
-\item[\it MMU\_MACHPHYS\_UPDATE:] update an entry in the
- machine-to-physical table. The calling domain must own the machine
- page in question (or be privileged).
-
-\item[\it MMU\_EXTENDED\_COMMAND:] perform additional MMU operations.
-The set of additional MMU operations is considerable, and includes
-updating {\tt cr3} (or just re-installing it for a TLB flush),
-flushing the cache, installing a new LDT, or pinning \& unpinning
-page-table pages (to ensure their reference count doesn't drop to zero
-which would require a revalidation of all entries).
-
-Further extended commands are used to deal with granting and
-acquiring page ownership; see Section~\ref{s:idc}.
-
-
-\end{description}
-
-More details on the precise format of all commands can be
-found in {\tt xen/include/public/xen.h}.
-
-
-\end{quote}
-
-Explicitly updating batches of page table entries is extremely
-efficient, but can require a number of alterations to the guest
-OS. Using the writable page table mode (Chapter~\ref{c:memory}) is
-recommended for new OS ports.
-
-Regardless of which page table update mode is being used, however,
-there are some occasions (notably handling a demand page fault) where
-a guest OS will wish to modify exactly one PTE rather than a
-batch. This is catered for by the following:
-
-\begin{quote}
-\hypercall{update\_va\_mapping(unsigned long page\_nr, unsigned long
-val, \\ unsigned long flags)}
-
-Update the currently installed PTE for the page {\tt page\_nr} to
-{\tt val}. As with {\tt mmu\_update()}, Xen checks the modification
-is safe before applying it. The {\tt flags} determine which kind
-of TLB flush, if any, should follow the update.
-
-\end{quote}
-
-Finally, sufficiently privileged domains may occasionally wish to manipulate
-the pages of others:
-\begin{quote}
-
-\hypercall{update\_va\_mapping\_otherdomain(unsigned long page\_nr,
-unsigned long val, unsigned long flags, uint16\_t domid)}
-
-Identical to {\tt update\_va\_mapping()} save that the pages being
-mapped must belong to the domain {\tt domid}.
-
-\end{quote}
-
-This privileged operation is currently used by backend virtual device
-drivers to safely map pages containing I/O data.
-
-
-
-\section{Segmentation Support}
-
-Xen allows guest OSes to install a custom GDT if they require it;
-this is context switched transparently whenever a domain is
-[de]scheduled. The following hypercall is effectively a
-`safe' version of {\tt lgdt}:
-
-\begin{quote}
-\hypercall{set\_gdt(unsigned long *frame\_list, int entries)}
-
-Install a global descriptor table for a domain; {\tt frame\_list} is
-an array of up to 16 machine page frames within which the GDT resides,
-with {\tt entries} being the actual number of descriptor-entry
-slots. All page frames must be mapped read-only within the guest's
-address space, and the table must be large enough to contain Xen's
-reserved entries (see {\tt xen/include/public/arch-x86\_32.h}).
-
-\end{quote}
-
-Many guest OSes will also wish to install LDTs; this is achieved by
-using {\tt mmu\_update()} with an extended command, passing the
-linear address of the LDT base along with the number of entries. No
-special safety checks are required; Xen needs to perform this task
-simply since {\tt lldt} requires CPL 0.
-
-
-Xen also allows guest operating systems to update just an
-individual segment descriptor in the GDT or LDT:
-
-\begin{quote}
-\hypercall{update\_descriptor(unsigned long ma, unsigned long word1,
-unsigned long word2)}
-
-Update the GDT/LDT entry at machine address {\tt ma}; the new
-8-byte descriptor is stored in {\tt word1} and {\tt word2}.
-Xen performs a number of checks to ensure the descriptor is
-valid.
-
-\end{quote}
-
-Guest OSes can use the above in place of context switching entire
-LDTs (or the GDT) when the number of changing descriptors is small.
-
-\section{Context Switching}
-
-When a guest OS wishes to context switch between two processes,
-it can use the page table and segmentation hypercalls described
-above to perform the the bulk of the privileged work. In addition,
-however, it will need to invoke Xen to switch the kernel (ring 1)
-stack pointer:
-
-\begin{quote}
-\hypercall{stack\_switch(unsigned long ss, unsigned long esp)}
-
-Request kernel stack switch from hypervisor; {\tt ss} is the new
-stack segment, which {\tt esp} is the new stack pointer.
-
-\end{quote}
-
-A final useful hypercall for context switching allows ``lazy''
-save and restore of floating point state:
-
-\begin{quote}
-\hypercall{fpu\_taskswitch(void)}
-
-This call instructs Xen to set the {\tt TS} bit in the {\tt cr0}
-control register; this means that the next attempt to use floating
-point will cause a trap which the guest OS can trap. Typically it will
-then save/restore the FP state, and clear the {\tt TS} bit.
-\end{quote}
-
-This is provided as an optimization only; guest OSes can also choose
-to save and restore FP state on all context switches for simplicity.
-
-
-\section{Physical Memory Management}
-
-As mentioned previously, each domain has a maximum and current
-memory allocation. The maximum allocation, set at domain creation
-time, cannot be modified. However a domain can choose to reduce
-and subsequently grow its current allocation by using the
-following call:
-
-\begin{quote}
-\hypercall{dom\_mem\_op(unsigned int op, unsigned long *extent\_list,
- unsigned long nr\_extents, unsigned int extent\_order)}
-
-Increase or decrease current memory allocation (as determined by
-the value of {\tt op}). Each invocation provides a list of
-extents each of which is $2^s$ pages in size,
-where $s$ is the value of {\tt extent\_order}.
-
-\end{quote}
-
-In addition to simply reducing or increasing the current memory
-allocation via a `balloon driver', this call is also useful for
-obtaining contiguous regions of machine memory when required (e.g.
-for certain PCI devices, or if using superpages).
-
-
-\section{Inter-Domain Communication}
-\label{s:idc}
-
-Xen provides a simple asynchronous notification mechanism via
-\emph{event channels}. Each domain has a set of end-points (or
-\emph{ports}) which may be bound to an event source (e.g. a physical
-IRQ, a virtual IRQ, or an port in another domain). When a pair of
-end-points in two different domains are bound together, then a `send'
-operation on one will cause an event to be received by the destination
-domain.
-
-The control and use of event channels involves the following hypercall:
-
-\begin{quote}
-\hypercall{event\_channel\_op(evtchn\_op\_t *op)}
-
-Inter-domain event-channel management; {\tt op} is a discriminated
-union which allows the following 7 operations:
-
-\begin{description}
-
-\item[\it alloc\_unbound:] allocate a free (unbound) local
- port and prepare for connection from a specified domain.
-\item[\it bind\_virq:] bind a local port to a virtual
-IRQ; any particular VIRQ can be bound to at most one port per domain.
-\item[\it bind\_pirq:] bind a local port to a physical IRQ;
-once more, a given pIRQ can be bound to at most one port per
-domain. Furthermore the calling domain must be sufficiently
-privileged.
-\item[\it bind\_interdomain:] construct an interdomain event
-channel; in general, the target domain must have previously allocated
-an unbound port for this channel, although this can be bypassed by
-privileged domains during domain setup.
-\item[\it close:] close an interdomain event channel.
-\item[\it send:] send an event to the remote end of a
-interdomain event channel.
-\item[\it status:] determine the current status of a local port.
-\end{description}
-
-For more details see
-{\tt xen/include/public/event\_channel.h}.
-
-\end{quote}
-
-Event channels are the fundamental communication primitive between
-Xen domains and seamlessly support SMP. However they provide little
-bandwidth for communication {\sl per se}, and hence are typically
-married with a piece of shared memory to produce effective and
-high-performance inter-domain communication.
-
-Safe sharing of memory pages between guest OSes is carried out by
-granting access on a per page basis to individual domains. This is
-achieved by using the {\tt grant\_table\_op()} hypercall.
-
-\begin{quote}
-\hypercall{grant\_table\_op(unsigned int cmd, void *uop, unsigned int count)}
-
-Grant or remove access to a particular page to a particular domain.
-
-\end{quote}
-
-This is not currently widely in use by guest operating systems, but
-we intend to integrate support more fully in the near future.
-
-\section{PCI Configuration}
-
-Domains with physical device access (i.e.\ driver domains) receive
-limited access to certain PCI devices (bus address space and
-interrupts). However many guest operating systems attempt to
-determine the PCI configuration by directly access the PCI BIOS,
-which cannot be allowed for safety.
-
-Instead, Xen provides the following hypercall:
-
-\begin{quote}
-\hypercall{physdev\_op(void *physdev\_op)}
-
-Perform a PCI configuration option; depending on the value
-of {\tt physdev\_op} this can be a PCI config read, a PCI config
-write, or a small number of other queries.
-
-\end{quote}
-
-
-For examples of using {\tt physdev\_op()}, see the
-Xen-specific PCI code in the linux sparse tree.
-
-\section{Administrative Operations}
-\label{s:dom0ops}
-
-A large number of control operations are available to a sufficiently
-privileged domain (typically domain 0). These allow the creation and
-management of new domains, for example. A complete list is given
-below: for more details on any or all of these, please see
-{\tt xen/include/public/dom0\_ops.h}
-
-
-\begin{quote}
-\hypercall{dom0\_op(dom0\_op\_t *op)}
-
-Administrative domain operations for domain management. The options are:
-
-\begin{description}
-\item [\it DOM0\_CREATEDOMAIN:] create a new domain
-
-\item [\it DOM0\_PAUSEDOMAIN:] remove a domain from the scheduler run
-queue.
-
-\item [\it DOM0\_UNPAUSEDOMAIN:] mark a paused domain as schedulable
- once again.
-
-\item [\it DOM0\_DESTROYDOMAIN:] deallocate all resources associated
-with a domain
-
-\item [\it DOM0\_GETMEMLIST:] get list of pages used by the domain
-
-\item [\it DOM0\_SCHEDCTL:]
-
-\item [\it DOM0\_ADJUSTDOM:] adjust scheduling priorities for domain
-
-\item [\it DOM0\_BUILDDOMAIN:] do final guest OS setup for domain
-
-\item [\it DOM0\_GETDOMAINFO:] get statistics about the domain
-
-\item [\it DOM0\_GETPAGEFRAMEINFO:]
-
-\item [\it DOM0\_GETPAGEFRAMEINFO2:]
-
-\item [\it DOM0\_IOPL:] set I/O privilege level
-
-\item [\it DOM0\_MSR:] read or write model specific registers
-
-\item [\it DOM0\_DEBUG:] interactively invoke the debugger
-
-\item [\it DOM0\_SETTIME:] set system time
-
-\item [\it DOM0\_READCONSOLE:] read console content from hypervisor buffer ring
-
-\item [\it DOM0\_PINCPUDOMAIN:] pin domain to a particular CPU
-
-\item [\it DOM0\_GETTBUFS:] get information about the size and location of
- the trace buffers (only on trace-buffer enabled builds)
-
-\item [\it DOM0\_PHYSINFO:] get information about the host machine
-
-\item [\it DOM0\_PCIDEV\_ACCESS:] modify PCI device access permissions
-
-\item [\it DOM0\_SCHED\_ID:] get the ID of the current Xen scheduler
-
-\item [\it DOM0\_SHADOW\_CONTROL:] switch between shadow page-table modes
-
-\item [\it DOM0\_SETDOMAININITIALMEM:] set initial memory allocation of a
domain
-
-\item [\it DOM0\_SETDOMAINMAXMEM:] set maximum memory allocation of a domain
-
-\item [\it DOM0\_SETDOMAINVMASSIST:] set domain VM assist options
-\end{description}
-\end{quote}
-
-Most of the above are best understood by looking at the code
-implementing them (in {\tt xen/common/dom0\_ops.c}) and in
-the user-space tools that use them (mostly in {\tt tools/libxc}).
-
-\section{Debugging Hypercalls}
-
-A few additional hypercalls are mainly useful for debugging:
-
-\begin{quote}
-\hypercall{console\_io(int cmd, int count, char *str)}
-
-Use Xen to interact with the console; operations are:
-
-{\it CONSOLEIO\_write}: Output count characters from buffer str.
-
-{\it CONSOLEIO\_read}: Input at most count characters into buffer str.
-\end{quote}
-
-A pair of hypercalls allows access to the underlying debug registers:
-\begin{quote}
-\hypercall{set\_debugreg(int reg, unsigned long value)}
-
-Set debug register {\tt reg} to {\tt value}
-
-\hypercall{get\_debugreg(int reg)}
-
-Return the contents of the debug register {\tt reg}
-\end{quote}
-
-And finally:
-\begin{quote}
-\hypercall{xen\_version(int cmd)}
-
-Request Xen version number.
-\end{quote}
-
-This is useful to ensure that user-space tools are in sync
-with the underlying hypervisor.
-
-\section{Deprecated Hypercalls}
-
-Xen is under constant development and refinement; as such there
-are plans to improve the way in which various pieces of functionality
-are exposed to guest OSes.
-
-\begin{quote}
-\hypercall{vm\_assist(unsigned int cmd, unsigned int type)}
-
-Toggle various memory management modes (in particular wrritable page
-tables and superpage support).
-
-\end{quote}
-
-This is likely to be replaced with mode values in the shared
-information page since this is more resilient for resumption
-after migration or checkpoint.
-
-
-
-
-
-
+%% chapter hypercalls moved to hypercalls.tex
+\include{src/interface/hypercalls}
%%
@@ -1173,279 +112,9 @@
%% new scheduler... not clear how many of them there are...
%%
-\begin{comment}
-
-\chapter{Scheduling API}
-
-The scheduling API is used by both the schedulers described above and should
-also be used by any new schedulers. It provides a generic interface and also
-implements much of the ``boilerplate'' code.
-
-Schedulers conforming to this API are described by the following
-structure:
-
-\begin{verbatim}
-struct scheduler
-{
- char *name; /* full name for this scheduler */
- char *opt_name; /* option name for this scheduler */
- unsigned int sched_id; /* ID for this scheduler */
-
- int (*init_scheduler) ();
- int (*alloc_task) (struct task_struct *);
- void (*add_task) (struct task_struct *);
- void (*free_task) (struct task_struct *);
- void (*rem_task) (struct task_struct *);
- void (*wake_up) (struct task_struct *);
- void (*do_block) (struct task_struct *);
- task_slice_t (*do_schedule) (s_time_t);
- int (*control) (struct sched_ctl_cmd *);
- int (*adjdom) (struct task_struct *,
- struct sched_adjdom_cmd *);
- s32 (*reschedule) (struct task_struct *);
- void (*dump_settings) (void);
- void (*dump_cpu_state) (int);
- void (*dump_runq_el) (struct task_struct *);
-};
-\end{verbatim}
-
-The only method that {\em must} be implemented is
-{\tt do\_schedule()}. However, if there is not some implementation for the
-{\tt wake\_up()} method then waking tasks will not get put on the runqueue!
-
-The fields of the above structure are described in more detail below.
-
-\subsubsection{name}
-
-The name field should point to a descriptive ASCII string.
-
-\subsubsection{opt\_name}
-
-This field is the value of the {\tt sched=} boot-time option that will select
-this scheduler.
-
-\subsubsection{sched\_id}
-
-This is an integer that uniquely identifies this scheduler. There should be a
-macro corrsponding to this scheduler ID in {\tt <xen/sched-if.h>}.
-
-\subsubsection{init\_scheduler}
-
-\paragraph*{Purpose}
-
-This is a function for performing any scheduler-specific initialisation. For
-instance, it might allocate memory for per-CPU scheduler data and initialise it
-appropriately.
-
-\paragraph*{Call environment}
-
-This function is called after the initialisation performed by the generic
-layer. The function is called exactly once, for the scheduler that has been
-selected.
-
-\paragraph*{Return values}
-
-This should return negative on failure --- this will cause an
-immediate panic and the system will fail to boot.
-
-\subsubsection{alloc\_task}
-
-\paragraph*{Purpose}
-Called when a {\tt task\_struct} is allocated by the generic scheduler
-layer. A particular scheduler implementation may use this method to
-allocate per-task data for this task. It may use the {\tt
-sched\_priv} pointer in the {\tt task\_struct} to point to this data.
-
-\paragraph*{Call environment}
-The generic layer guarantees that the {\tt sched\_priv} field will
-remain intact from the time this method is called until the task is
-deallocated (so long as the scheduler implementation does not change
-it explicitly!).
-
-\paragraph*{Return values}
-Negative on failure.
-
-\subsubsection{add\_task}
-
-\paragraph*{Purpose}
-
-Called when a task is initially added by the generic layer.
-
-\paragraph*{Call environment}
-
-The fields in the {\tt task\_struct} are now filled out and available for use.
-Schedulers should implement appropriate initialisation of any per-task private
-information in this method.
-
-\subsubsection{free\_task}
-
-\paragraph*{Purpose}
-
-Schedulers should free the space used by any associated private data
-structures.
-
-\paragraph*{Call environment}
-
-This is called when a {\tt task\_struct} is about to be deallocated.
-The generic layer will have done generic task removal operations and
-(if implemented) called the scheduler's {\tt rem\_task} method before
-this method is called.
-
-\subsubsection{rem\_task}
-
-\paragraph*{Purpose}
-
-This is called when a task is being removed from scheduling (but is
-not yet being freed).
-
-\subsubsection{wake\_up}
-
-\paragraph*{Purpose}
-
-Called when a task is woken up, this method should put the task on the runqueue
-(or do the scheduler-specific equivalent action).
-
-\paragraph*{Call environment}
-
-The task is already set to state RUNNING.
-
-\subsubsection{do\_block}
-
-\paragraph*{Purpose}
-
-This function is called when a task is blocked. This function should
-not remove the task from the runqueue.
-
-\paragraph*{Call environment}
-
-The EVENTS\_MASTER\_ENABLE\_BIT is already set and the task state changed to
-TASK\_INTERRUPTIBLE on entry to this method. A call to the {\tt
- do\_schedule} method will be made after this method returns, in
-order to select the next task to run.
-
-\subsubsection{do\_schedule}
-
-This method must be implemented.
-
-\paragraph*{Purpose}
-
-The method is called each time a new task must be chosen for scheduling on the
-current CPU. The current time as passed as the single argument (the current
-task can be found using the {\tt current} macro).
-
-This method should select the next task to run on this CPU and set it's minimum
-time to run as well as returning the data described below.
-
-This method should also take the appropriate action if the previous
-task has blocked, e.g. removing it from the runqueue.
-
-\paragraph*{Call environment}
-
-The other fields in the {\tt task\_struct} are updated by the generic layer,
-which also performs all Xen-specific tasks and performs the actual task switch
-(unless the previous task has been chosen again).
-
-This method is called with the {\tt schedule\_lock} held for the current CPU
-and local interrupts disabled.
-
-\paragraph*{Return values}
-
-Must return a {\tt struct task\_slice} describing what task to run and how long
-for (at maximum).
-
-\subsubsection{control}
-
-\paragraph*{Purpose}
-
-This method is called for global scheduler control operations. It takes a
-pointer to a {\tt struct sched\_ctl\_cmd}, which it should either
-source data from or populate with data, depending on the value of the
-{\tt direction} field.
-
-\paragraph*{Call environment}
-
-The generic layer guarantees that when this method is called, the
-caller selected the correct scheduler ID, hence the scheduler's
-implementation does not need to sanity-check these parts of the call.
-
-\paragraph*{Return values}
-
-This function should return the value to be passed back to user space, hence it
-should either be 0 or an appropriate errno value.
-
-\subsubsection{sched\_adjdom}
-
-\paragraph*{Purpose}
-
-This method is called to adjust the scheduling parameters of a particular
-domain, or to query their current values. The function should check
-the {\tt direction} field of the {\tt sched\_adjdom\_cmd} it receives in
-order to determine which of these operations is being performed.
-
-\paragraph*{Call environment}
-
-The generic layer guarantees that the caller has specified the correct
-control interface version and scheduler ID and that the supplied {\tt
-task\_struct} will not be deallocated during the call (hence it is not
-necessary to {\tt get\_task\_struct}).
-
-\paragraph*{Return values}
-
-This function should return the value to be passed back to user space, hence it
-should either be 0 or an appropriate errno value.
-
-\subsubsection{reschedule}
-
-\paragraph*{Purpose}
-
-This method is called to determine if a reschedule is required as a result of a
-particular task.
-
-\paragraph*{Call environment}
-The generic layer will cause a reschedule if the current domain is the idle
-task or it has exceeded its minimum time slice before a reschedule. The
-generic layer guarantees that the task passed is not currently running but is
-on the runqueue.
-
-\paragraph*{Return values}
-
-Should return a mask of CPUs to cause a reschedule on.
-
-\subsubsection{dump\_settings}
-
-\paragraph*{Purpose}
-
-If implemented, this should dump any private global settings for this
-scheduler to the console.
-
-\paragraph*{Call environment}
-
-This function is called with interrupts enabled.
-
-\subsubsection{dump\_cpu\_state}
-
-\paragraph*{Purpose}
-
-This method should dump any private settings for the specified CPU.
-
-\paragraph*{Call environment}
-
-This function is called with interrupts disabled and the {\tt schedule\_lock}
-for the specified CPU held.
-
-\subsubsection{dump\_runq\_el}
-
-\paragraph*{Purpose}
-
-This method should dump any private settings for the specified task.
-
-\paragraph*{Call environment}
-
-This function is called with interrupts disabled and the {\tt schedule\_lock}
-for the task's CPU held.
-
-\end{comment}
-
+%% \include{src/interface/scheduling}
+%% scheduling information moved to scheduling.tex
+%% still commented out
@@ -1457,74 +126,9 @@
%% (and/or kip's stuff?) and write about that instead?
%%
-\begin{comment}
-
-\chapter{Debugging}
-
-Xen provides tools for debugging both Xen and guest OSes. Currently, the
-Pervasive Debugger provides a GDB stub, which provides facilities for symbolic
-debugging of Xen itself and of OS kernels running on top of Xen. The Trace
-Buffer provides a lightweight means to log data about Xen's internal state and
-behaviour at runtime, for later analysis.
-
-\section{Pervasive Debugger}
-
-Information on using the pervasive debugger is available in pdb.txt.
-
-
-\section{Trace Buffer}
-
-The trace buffer provides a means to observe Xen's operation from domain 0.
-Trace events, inserted at key points in Xen's code, record data that can be
-read by the {\tt xentrace} tool. Recording these events has a low overhead
-and hence the trace buffer may be useful for debugging timing-sensitive
-behaviours.
-
-\subsection{Internal API}
-
-To use the trace buffer functionality from within Xen, you must {\tt \#include
-<xen/trace.h>}, which contains definitions related to the trace buffer. Trace
-events are inserted into the buffer using the {\tt TRACE\_xD} ({\tt x} = 0, 1,
-2, 3, 4 or 5) macros. These all take an event number, plus {\tt x} additional
-(32-bit) data as their arguments. For trace buffer-enabled builds of Xen these
-will insert the event ID and data into the trace buffer, along with the current
-value of the CPU cycle-counter. For builds without the trace buffer enabled,
-the macros expand to no-ops and thus can be left in place without incurring
-overheads.
-
-\subsection{Trace-enabled builds}
-
-By default, the trace buffer is enabled only in debug builds (i.e. {\tt NDEBUG}
-is not defined). It can be enabled separately by defining {\tt TRACE\_BUFFER},
-either in {\tt <xen/config.h>} or on the gcc command line.
-
-The size (in pages) of the per-CPU trace buffers can be specified using the
-{\tt tbuf\_size=n } boot parameter to Xen. If the size is set to 0, the trace
-buffers will be disabled.
-
-\subsection{Dumping trace data}
-
-When running a trace buffer build of Xen, trace data are written continuously
-into the buffer data areas, with newer data overwriting older data. This data
-can be captured using the {\tt xentrace} program in domain 0.
-
-The {\tt xentrace} tool uses {\tt /dev/mem} in domain 0 to map the trace
-buffers into its address space. It then periodically polls all the buffers for
-new data, dumping out any new records from each buffer in turn. As a result,
-for machines with multiple (logical) CPUs, the trace buffer output will not be
-in overall chronological order.
-
-The output from {\tt xentrace} can be post-processed using {\tt
-xentrace\_cpusplit} (used to split trace data out into per-cpu log files) and
-{\tt xentrace\_format} (used to pretty-print trace data). For the predefined
-trace points, there is an example format file in {\tt tools/xentrace/formats }.
-
-For more information, see the manual pages for {\tt xentrace}, {\tt
-xentrace\_format} and {\tt xentrace\_cpusplit}.
-
-\end{comment}
-
-
+%% \include{src/interface/debugging}
+%% debugging information moved to debugging.tex
+%% still commented out
\end{document}
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user.tex
--- a/docs/src/user.tex Tue Sep 20 09:08:26 2005
+++ b/docs/src/user.tex Tue Sep 20 09:17:33 2005
@@ -59,1803 +59,36 @@
\renewcommand{\floatpagefraction}{.8}
\setstretch{1.1}
+
\part{Introduction and Tutorial}
-\chapter{Introduction}
-
-Xen is a {\em paravirtualising} virtual machine monitor (VMM), or
-`hypervisor', for the x86 processor architecture. Xen can securely
-execute multiple virtual machines on a single physical system with
-close-to-native performance. The virtual machine technology
-facilitates enterprise-grade functionality, including:
-
-\begin{itemize}
-\item Virtual machines with performance close to native
- hardware.
-\item Live migration of running virtual machines between physical hosts.
-\item Excellent hardware support (supports most Linux device drivers).
-\item Sandboxed, restartable device drivers.
-\end{itemize}
-
-Paravirtualisation permits very high performance virtualisation,
-even on architectures like x86 that are traditionally
-very hard to virtualise.
-The drawback of this approach is that it requires operating systems to
-be {\em ported} to run on Xen. Porting an OS to run on Xen is similar
-to supporting a new hardware platform, however the process
-is simplified because the paravirtual machine architecture is very
-similar to the underlying native hardware. Even though operating system
-kernels must explicitly support Xen, a key feature is that user space
-applications and libraries {\em do not} require modification.
-
-Xen support is available for increasingly many operating systems:
-right now, Linux 2.4, Linux 2.6 and NetBSD are available for Xen 2.0.
-A FreeBSD port is undergoing testing and will be incorporated into the
-release soon. Other OS ports, including Plan 9, are in progress. We
-hope that that arch-xen patches will be incorporated into the
-mainstream releases of these operating systems in due course (as has
-already happened for NetBSD).
-
-Possible usage scenarios for Xen include:
-\begin{description}
-\item [Kernel development.] Test and debug kernel modifications in a
- sandboxed virtual machine --- no need for a separate test
- machine.
-\item [Multiple OS configurations.] Run multiple operating systems
- simultaneously, for instance for compatibility or QA purposes.
-\item [Server consolidation.] Move multiple servers onto a single
- physical host with performance and fault isolation provided at
- virtual machine boundaries.
-\item [Cluster computing.] Management at VM granularity provides more
- flexibility than separately managing each physical host, but
- better control and isolation than single-system image solutions,
- particularly by using live migration for load balancing.
-\item [Hardware support for custom OSes.] Allow development of new OSes
- while benefiting from the wide-ranging hardware support of
- existing OSes such as Linux.
-\end{description}
-
-\section{Structure of a Xen-Based System}
-
-A Xen system has multiple layers, the lowest and most privileged of
-which is Xen itself.
-Xen in turn may host multiple {\em guest} operating systems, each of
-which is executed within a secure virtual machine (in Xen terminology,
-a {\em domain}). Domains are scheduled by Xen to make effective use of
-the available physical CPUs. Each guest OS manages its own
-applications, which includes responsibility for scheduling each
-application within the time allotted to the VM by Xen.
-
-The first domain, {\em domain 0}, is created automatically when the
-system boots and has special management privileges. Domain 0 builds
-other domains and manages their virtual devices. It also performs
-administrative tasks such as suspending, resuming and migrating other
-virtual machines.
-
-Within domain 0, a process called \emph{xend} runs to manage the system.
-\Xend is responsible for managing virtual machines and providing access
-to their consoles. Commands are issued to \xend over an HTTP
-interface, either from a command-line tool or from a web browser.
-
-\section{Hardware Support}
-
-Xen currently runs only on the x86 architecture, requiring a `P6' or
-newer processor (e.g. Pentium Pro, Celeron, Pentium II, Pentium III,
-Pentium IV, Xeon, AMD Athlon, AMD Duron). Multiprocessor machines are
-supported, and we also have basic support for HyperThreading (SMT),
-although this remains a topic for ongoing research. A port
-specifically for x86/64 is in progress, although Xen already runs on
-such systems in 32-bit legacy mode. In addition a port to the IA64
-architecture is approaching completion. We hope to add other
-architectures such as PPC and ARM in due course.
-
-
-Xen can currently use up to 4GB of memory. It is possible for x86
-machines to address up to 64GB of physical memory but there are no
-current plans to support these systems: The x86/64 port is the
-planned route to supporting larger memory sizes.
-
-Xen offloads most of the hardware support issues to the guest OS
-running in Domain~0. Xen itself contains only the code required to
-detect and start secondary processors, set up interrupt routing, and
-perform PCI bus enumeration. Device drivers run within a privileged
-guest OS rather than within Xen itself. This approach provides
-compatibility with the majority of device hardware supported by Linux.
-The default XenLinux build contains support for relatively modern
-server-class network and disk hardware, but you can add support for
-other hardware by configuring your XenLinux kernel in the normal way.
-
-\section{History}
-
-Xen was originally developed by the Systems Research Group at the
-University of Cambridge Computer Laboratory as part of the XenoServers
-project, funded by the UK-EPSRC.
-XenoServers aim to provide a `public infrastructure for
-global distributed computing', and Xen plays a key part in that,
-allowing us to efficiently partition a single machine to enable
-multiple independent clients to run their operating systems and
-applications in an environment providing protection, resource
-isolation and accounting. The project web page contains further
-information along with pointers to papers and technical reports:
-\path{http://www.cl.cam.ac.uk/xeno}
-
-Xen has since grown into a fully-fledged project in its own right,
-enabling us to investigate interesting research issues regarding the
-best techniques for virtualising resources such as the CPU, memory,
-disk and network. The project has been bolstered by support from
-Intel Research Cambridge, and HP Labs, who are now working closely
-with us.
-
-Xen was first described in a paper presented at SOSP in
-2003\footnote{\tt
-http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf}, and the first
-public release (1.0) was made that October. Since then, Xen has
-significantly matured and is now used in production scenarios on
-many sites.
-
-Xen 2.0 features greatly enhanced hardware support, configuration
-flexibility, usability and a larger complement of supported operating
-systems. This latest release takes Xen a step closer to becoming the
-definitive open source solution for virtualisation.
-
-\chapter{Installation}
-
-The Xen distribution includes three main components: Xen itself, ports
-of Linux 2.4 and 2.6 and NetBSD to run on Xen, and the user-space
-tools required to manage a Xen-based system. This chapter describes
-how to install the Xen 2.0 distribution from source. Alternatively,
-there may be pre-built packages available as part of your operating
-system distribution.
-
-\section{Prerequisites}
-\label{sec:prerequisites}
-
-The following is a full list of prerequisites. Items marked `$\dag$'
-are required by the \xend control tools, and hence required if you
-want to run more than one virtual machine; items marked `$*$' are only
-required if you wish to build from source.
-\begin{itemize}
-\item A working Linux distribution using the GRUB bootloader and
-running on a P6-class (or newer) CPU.
-\item [$\dag$] The \path{iproute2} package.
-\item [$\dag$] The Linux bridge-utils\footnote{Available from
-{\tt http://bridge.sourceforge.net}} (e.g., \path{/sbin/brctl})
-\item [$\dag$] An installation of Twisted v1.3 or
-above\footnote{Available from {\tt
-http://www.twistedmatrix.com}}. There may be a binary package
-available for your distribution; alternatively it can be installed by
-running `{\sl make install-twisted}' in the root of the Xen source
-tree.
-\item [$*$] Build tools (gcc v3.2.x or v3.3.x, binutils, GNU make).
-\item [$*$] Development installation of libcurl (e.g., libcurl-devel)
-\item [$*$] Development installation of zlib (e.g., zlib-dev).
-\item [$*$] Development installation of Python v2.2 or later (e.g.,
python-dev).
-\item [$*$] \LaTeX and transfig are required to build the documentation.
-\end{itemize}
-
-Once you have satisfied the relevant prerequisites, you can
-now install either a binary or source distribution of Xen.
-
-\section{Installing from Binary Tarball}
-
-Pre-built tarballs are available for download from the Xen
-download page
-\begin{quote}
-{\tt http://xen.sf.net}
-\end{quote}
-
-Once you've downloaded the tarball, simply unpack and install:
-\begin{verbatim}
-# tar zxvf xen-2.0-install.tgz
-# cd xen-2.0-install
-# sh ./install.sh
-\end{verbatim}
-
-Once you've installed the binaries you need to configure
-your system as described in Section~\ref{s:configure}.
-
-\section{Installing from Source}
-
-This section describes how to obtain, build, and install
-Xen from source.
-
-\subsection{Obtaining the Source}
-
-The Xen source tree is available as either a compressed source tar
-ball or as a clone of our master BitKeeper repository.
-
-\begin{description}
-\item[Obtaining the Source Tarball]\mbox{} \\
-Stable versions (and daily snapshots) of the Xen source tree are
-available as compressed tarballs from the Xen download page
-\begin{quote}
-{\tt http://xen.sf.net}
-\end{quote}
-
-\item[Using BitKeeper]\mbox{} \\
-If you wish to install Xen from a clone of our latest BitKeeper
-repository then you will need to install the BitKeeper tools.
-Download instructions for BitKeeper can be obtained by filling out the
-form at:
-
-\begin{quote}
-{\tt http://www.bitmover.com/cgi-bin/download.cgi}
-\end{quote}
-The public master BK repository for the 2.0 release lives at:
-\begin{quote}
-{\tt bk://xen.bkbits.net/xen-2.0.bk}
-\end{quote}
-You can use BitKeeper to
-download it and keep it updated with the latest features and fixes.
-
-Change to the directory in which you want to put the source code, then
-run:
-\begin{verbatim}
-# bk clone bk://xen.bkbits.net/xen-2.0.bk
-\end{verbatim}
-
-Under your current directory, a new directory named \path{xen-2.0.bk}
-has been created, which contains all the source code for Xen, the OS
-ports, and the control tools. You can update your repository with the
-latest changes at any time by running:
-\begin{verbatim}
-# cd xen-2.0.bk # to change into the local repository
-# bk pull # to update the repository
-\end{verbatim}
-\end{description}
-
-%\section{The distribution}
-%
-%The Xen source code repository is structured as follows:
-%
-%\begin{description}
-%\item[\path{tools/}] Xen node controller daemon (Xend), command line tools,
-% control libraries
-%\item[\path{xen/}] The Xen VMM.
-%\item[\path{linux-*-xen-sparse/}] Xen support for Linux.
-%\item[\path{linux-*-patches/}] Experimental patches for Linux.
-%\item[\path{netbsd-*-xen-sparse/}] Xen support for NetBSD.
-%\item[\path{docs/}] Various documentation files for users and developers.
-%\item[\path{extras/}] Bonus extras.
-%\end{description}
-
-\subsection{Building from Source}
-
-The top-level Xen Makefile includes a target `world' that will do the
-following:
-
-\begin{itemize}
-\item Build Xen
-\item Build the control tools, including \xend
-\item Download (if necessary) and unpack the Linux 2.6 source code,
- and patch it for use with Xen
-\item Build a Linux kernel to use in domain 0 and a smaller
- unprivileged kernel, which can optionally be used for
- unprivileged virtual machines.
-\end{itemize}
-
-
-After the build has completed you should have a top-level
-directory called \path{dist/} in which all resulting targets
-will be placed; of particular interest are the two kernels
-XenLinux kernel images, one with a `-xen0' extension
-which contains hardware device drivers and drivers for Xen's virtual
-devices, and one with a `-xenU' extension that just contains the
-virtual ones. These are found in \path{dist/install/boot/} along
-with the image for Xen itself and the configuration files used
-during the build.
-
-The NetBSD port can be built using:
-\begin{quote}
-\begin{verbatim}
-# make netbsd20
-\end{verbatim}
-\end{quote}
-NetBSD port is built using a snapshot of the netbsd-2-0 cvs branch.
-The snapshot is downloaded as part of the build process, if it is not
-yet present in the \path{NETBSD\_SRC\_PATH} search path. The build
-process also downloads a toolchain which includes all the tools
-necessary to build the NetBSD kernel under Linux.
-
-To customize further the set of kernels built you need to edit
-the top-level Makefile. Look for the line:
-
-\begin{quote}
-\begin{verbatim}
-KERNELS ?= mk.linux-2.6-xen0 mk.linux-2.6-xenU
-\end{verbatim}
-\end{quote}
-
-You can edit this line to include any set of operating system kernels
-which have configurations in the top-level \path{buildconfigs/}
-directory, for example \path{mk.linux-2.4-xenU} to build a Linux 2.4
-kernel containing only virtual device drivers.
-
-%% Inspect the Makefile if you want to see what goes on during a build.
-%% Building Xen and the tools is straightforward, but XenLinux is more
-%% complicated. The makefile needs a `pristine' Linux kernel tree to which
-%% it will then add the Xen architecture files. You can tell the
-%% makefile the location of the appropriate Linux compressed tar file by
-%% setting the LINUX\_SRC environment variable, e.g. \\
-%% \verb!# LINUX_SRC=/tmp/linux-2.6.11.tar.bz2 make world! \\ or by
-%% placing the tar file somewhere in the search path of {\tt
-%% LINUX\_SRC\_PATH} which defaults to `{\tt .:..}'. If the makefile
-%% can't find a suitable kernel tar file it attempts to download it from
-%% kernel.org (this won't work if you're behind a firewall).
-
-%% After untaring the pristine kernel tree, the makefile uses the {\tt
-%% mkbuildtree} script to add the Xen patches to the kernel.
-
-
-%% The procedure is similar to build the Linux 2.4 port: \\
-%% \verb!# LINUX_SRC=/path/to/linux2.4/source make linux24!
-
-
-%% \framebox{\parbox{5in}{
-%% {\bf Distro specific:} \\
-%% {\it Gentoo} --- if not using udev (most installations, currently), you'll
need
-%% to enable devfs and devfs mount at boot time in the xen0 config.
-%% }}
-
-\subsection{Custom XenLinux Builds}
-
-% If you have an SMP machine you may wish to give the {\tt '-j4'}
-% argument to make to get a parallel build.
-
-If you wish to build a customized XenLinux kernel (e.g. to support
-additional devices or enable distribution-required features), you can
-use the standard Linux configuration mechanisms, specifying that the
-architecture being built for is \path{xen}, e.g:
-\begin{quote}
-\begin{verbatim}
-# cd linux-2.6.11-xen0
-# make ARCH=xen xconfig
-# cd ..
-# make
-\end{verbatim}
-\end{quote}
-
-You can also copy an existing Linux configuration (\path{.config})
-into \path{linux-2.6.11-xen0} and execute:
-\begin{quote}
-\begin{verbatim}
-# make ARCH=xen oldconfig
-\end{verbatim}
-\end{quote}
-
-You may be prompted with some Xen-specific options; we
-advise accepting the defaults for these options.
-
-Note that the only difference between the two types of Linux kernel
-that are built is the configuration file used for each. The "U"
-suffixed (unprivileged) versions don't contain any of the physical
-hardware device drivers, leading to a 30\% reduction in size; hence
-you may prefer these for your non-privileged domains. The `0'
-suffixed privileged versions can be used to boot the system, as well
-as in driver domains and unprivileged domains.
-
-
-\subsection{Installing the Binaries}
-
-
-The files produced by the build process are stored under the
-\path{dist/install/} directory. To install them in their default
-locations, do:
-\begin{quote}
-\begin{verbatim}
-# make install
-\end{verbatim}
-\end{quote}
-
-
-Alternatively, users with special installation requirements may wish
-to install them manually by copying the files to their appropriate
-destinations.
-
-%% Files in \path{install/boot/} include:
-%% \begin{itemize}
-%% \item \path{install/boot/xen-2.0.gz} Link to the Xen 'kernel'
-%% \item \path{install/boot/vmlinuz-2.6-xen0} Link to domain 0 XenLinux kernel
-%% \item \path{install/boot/vmlinuz-2.6-xenU} Link to unprivileged XenLinux
kernel
-%% \end{itemize}
-
-The \path{dist/install/boot} directory will also contain the config files
-used for building the XenLinux kernels, and also versions of Xen and
-XenLinux kernels that contain debug symbols (\path{xen-syms-2.0.6} and
-\path{vmlinux-syms-2.6.11.11-xen0}) which are essential for interpreting crash
-dumps. Retain these files as the developers may wish to see them if
-you post on the mailing list.
-
-
-
-
-
-\section{Configuration}
-\label{s:configure}
-Once you have built and installed the Xen distribution, it is
-simple to prepare the machine for booting and running Xen.
-
-\subsection{GRUB Configuration}
-
-An entry should be added to \path{grub.conf} (often found under
-\path{/boot/} or \path{/boot/grub/}) to allow Xen / XenLinux to boot.
-This file is sometimes called \path{menu.lst}, depending on your
-distribution. The entry should look something like the following:
-
-{\small
-\begin{verbatim}
-title Xen 2.0 / XenLinux 2.6
- kernel /boot/xen-2.0.gz dom0_mem=131072
- module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro console=tty0
-\end{verbatim}
-}
-
-The kernel line tells GRUB where to find Xen itself and what boot
-parameters should be passed to it (in this case, setting domain 0's
-memory allocation in kilobytes and the settings for the serial port). For more
-details on the various Xen boot parameters see Section~\ref{s:xboot}.
-
-The module line of the configuration describes the location of the
-XenLinux kernel that Xen should start and the parameters that should
-be passed to it (these are standard Linux parameters, identifying the
-root device and specifying it be initially mounted read only and
-instructing that console output be sent to the screen). Some
-distributions such as SuSE do not require the \path{ro} parameter.
-
-%% \framebox{\parbox{5in}{
-%% {\bf Distro specific:} \\
-%% {\it SuSE} --- Omit the {\tt ro} option from the XenLinux kernel
-%% command line, since the partition won't be remounted rw during boot.
-%% }}
-
-
-If you want to use an initrd, just add another \path{module} line to
-the configuration, as usual:
-{\small
-\begin{verbatim}
- module /boot/my_initrd.gz
-\end{verbatim}
-}
-
-As always when installing a new kernel, it is recommended that you do
-not delete existing menu options from \path{menu.lst} --- you may want
-to boot your old Linux kernel in future, particularly if you
-have problems.
-
-
-\subsection{Serial Console (optional)}
-
-%% kernel /boot/xen-2.0.gz dom0_mem=131072 com1=115200,8n1
-%% module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro
-
-
-In order to configure Xen serial console output, it is necessary to add
-an boot option to your GRUB config; e.g. replace the above kernel line
-with:
-\begin{quote}
-{\small
-\begin{verbatim}
- kernel /boot/xen.gz dom0_mem=131072 com1=115200,8n1
-\end{verbatim}}
-\end{quote}
-
-This configures Xen to output on COM1 at 115,200 baud, 8 data bits,
-1 stop bit and no parity. Modify these parameters for your set up.
-
-One can also configure XenLinux to share the serial console; to
-achieve this append ``\path{console=ttyS0}'' to your
-module line.
-
-
-If you wish to be able to log in over the XenLinux serial console it
-is necessary to add a line into \path{/etc/inittab}, just as per
-regular Linux. Simply add the line:
-\begin{quote}
-{\small
-{\tt c:2345:respawn:/sbin/mingetty ttyS0}
-}
-\end{quote}
-
-and you should be able to log in. Note that to successfully log in
-as root over the serial line will require adding \path{ttyS0} to
-\path{/etc/securetty} in most modern distributions.
-
-\subsection{TLS Libraries}
-
-Users of the XenLinux 2.6 kernel should disable Thread Local Storage
-(e.g.\ by doing a \path{mv /lib/tls /lib/tls.disabled}) before
-attempting to run with a XenLinux kernel\footnote{If you boot without first
-disabling TLS, you will get a warning message during the boot
-process. In this case, simply perform the rename after the machine is
-up and then run \texttt{/sbin/ldconfig} to make it take effect.}. You can
-always reenable it by restoring the directory to its original location
-(i.e.\ \path{mv /lib/tls.disabled /lib/tls}).
-
-The reason for this is that the current TLS implementation uses
-segmentation in a way that is not permissible under Xen. If TLS is
-not disabled, an emulation mode is used within Xen which reduces
-performance substantially.
-
-We hope that this issue can be resolved by working with Linux
-distribution vendors to implement a minor backward-compatible change
-to the TLS library.
-
-\section{Booting Xen}
-
-It should now be possible to restart the system and use Xen. Reboot
-as usual but choose the new Xen option when the Grub screen appears.
-
-What follows should look much like a conventional Linux boot. The
-first portion of the output comes from Xen itself, supplying low level
-information about itself and the machine it is running on. The
-following portion of the output comes from XenLinux.
-
-You may see some errors during the XenLinux boot. These are not
-necessarily anything to worry about --- they may result from kernel
-configuration differences between your XenLinux kernel and the one you
-usually use.
-
-When the boot completes, you should be able to log into your system as
-usual. If you are unable to log in to your system running Xen, you
-should still be able to reboot with your normal Linux kernel.
-
-
-\chapter{Starting Additional Domains}
-
-The first step in creating a new domain is to prepare a root
-filesystem for it to boot off. Typically, this might be stored in a
-normal partition, an LVM or other volume manager partition, a disk
-file or on an NFS server. A simple way to do this is simply to boot
-from your standard OS install CD and install the distribution into
-another partition on your hard drive.
-
-To start the \xend control daemon, type
-\begin{quote}
-\verb!# xend start!
-\end{quote}
-If you
-wish the daemon to start automatically, see the instructions in
-Section~\ref{s:xend}. Once the daemon is running, you can use the
-\path{xm} tool to monitor and maintain the domains running on your
-system. This chapter provides only a brief tutorial: we provide full
-details of the \path{xm} tool in the next chapter.
-
-%\section{From the web interface}
-%
-%Boot the Xen machine and start Xensv (see Chapter~\ref{cha:xensv} for
-%more details) using the command: \\
-%\verb_# xensv start_ \\
-%This will also start Xend (see Chapter~\ref{cha:xend} for more information).
-%
-%The domain management interface will then be available at {\tt
-%http://your\_machine:8080/}. This provides a user friendly wizard for
-%starting domains and functions for managing running domains.
-%
-%\section{From the command line}
-
-
-\section{Creating a Domain Configuration File}
-
-Before you can start an additional domain, you must create a
-configuration file. We provide two example files which you
-can use as a starting point:
-\begin{itemize}
- \item \path{/etc/xen/xmexample1} is a simple template configuration file
- for describing a single VM.
-
- \item \path{/etc/xen/xmexample2} file is a template description that
- is intended to be reused for multiple virtual machines. Setting
- the value of the \path{vmid} variable on the \path{xm} command line
- fills in parts of this template.
-\end{itemize}
-
-Copy one of these files and edit it as appropriate.
-Typical values you may wish to edit include:
-
-\begin{quote}
-\begin{description}
-\item[kernel] Set this to the path of the kernel you compiled for use
- with Xen (e.g.\ \path{kernel = '/boot/vmlinuz-2.6-xenU'})
-\item[memory] Set this to the size of the domain's memory in
-megabytes (e.g.\ \path{memory = 64})
-\item[disk] Set the first entry in this list to calculate the offset
-of the domain's root partition, based on the domain ID. Set the
-second to the location of \path{/usr} if you are sharing it between
-domains (e.g.\ \path{disk = ['phy:your\_hard\_drive\%d,sda1,w' \%
-(base\_partition\_number + vmid), 'phy:your\_usr\_partition,sda6,r' ]}
-\item[dhcp] Uncomment the dhcp variable, so that the domain will
-receive its IP address from a DHCP server (e.g.\ \path{dhcp='dhcp'})
-\end{description}
-\end{quote}
-
-You may also want to edit the {\bf vif} variable in order to choose
-the MAC address of the virtual ethernet interface yourself. For
-example:
-\begin{quote}
-\verb_vif = ['mac=00:06:AA:F6:BB:B3']_
-\end{quote}
-If you do not set this variable, \xend will automatically generate a
-random MAC address from an unused range.
-
-
-\section{Booting the Domain}
-
-The \path{xm} tool provides a variety of commands for managing domains.
-Use the \path{create} command to start new domains. Assuming you've
-created a configuration file \path{myvmconf} based around
-\path{/etc/xen/xmexample2}, to start a domain with virtual
-machine ID~1 you should type:
-
-\begin{quote}
-\begin{verbatim}
-# xm create -c myvmconf vmid=1
-\end{verbatim}
-\end{quote}
-
-
-The \path{-c} switch causes \path{xm} to turn into the domain's
-console after creation. The \path{vmid=1} sets the \path{vmid}
-variable used in the \path{myvmconf} file.
-
-
-You should see the console boot messages from the new domain
-appearing in the terminal in which you typed the command,
-culminating in a login prompt.
-
-
-\section{Example: ttylinux}
-
-Ttylinux is a very small Linux distribution, designed to require very
-few resources. We will use it as a concrete example of how to start a
-Xen domain. Most users will probably want to install a full-featured
-distribution once they have mastered the basics\footnote{ttylinux is
-maintained by Pascal Schmidt. You can download source packages from
-the distribution's home page: {\tt http://www.minimalinux.org/ttylinux/}}.
-
-\begin{enumerate}
-\item Download and extract the ttylinux disk image from the Files
-section of the project's SourceForge site (see
-\path{http://sf.net/projects/xen/}).
-\item Create a configuration file like the following:
-\begin{verbatim}
-kernel = "/boot/vmlinuz-2.6-xenU"
-memory = 64
-name = "ttylinux"
-nics = 1
-ip = "1.2.3.4"
-disk = ['file:/path/to/ttylinux/rootfs,sda1,w']
-root = "/dev/sda1 ro"
-\end{verbatim}
-\item Now start the domain and connect to its console:
-\begin{verbatim}
-xm create configfile -c
-\end{verbatim}
-\item Login as root, password root.
-\end{enumerate}
-
-
-\section{Starting / Stopping Domains Automatically}
-
-It is possible to have certain domains start automatically at boot
-time and to have dom0 wait for all running domains to shutdown before
-it shuts down the system.
-
-To specify a domain is to start at boot-time, place its
-configuration file (or a link to it) under \path{/etc/xen/auto/}.
-
-A Sys-V style init script for RedHat and LSB-compliant systems is
-provided and will be automatically copied to \path{/etc/init.d/}
-during install. You can then enable it in the appropriate way for
-your distribution.
-
-For instance, on RedHat:
-
-\begin{quote}
-\verb_# chkconfig --add xendomains_
-\end{quote}
-
-By default, this will start the boot-time domains in runlevels 3, 4
-and 5.
-
-You can also use the \path{service} command to run this script
-manually, e.g:
-
-\begin{quote}
-\verb_# service xendomains start_
-
-Starts all the domains with config files under /etc/xen/auto/.
-\end{quote}
-
-
-\begin{quote}
-\verb_# service xendomains stop_
-
-Shuts down ALL running Xen domains.
-\end{quote}
-
-\chapter{Domain Management Tools}
-
-The previous chapter described a simple example of how to configure
-and start a domain. This chapter summarises the tools available to
-manage running domains.
-
-\section{Command-line Management}
-
-Command line management tasks are also performed using the \path{xm}
-tool. For online help for the commands available, type:
-\begin{quote}
-\verb_# xm help_
-\end{quote}
-
-You can also type \path{xm help $<$command$>$} for more information
-on a given command.
-
-\subsection{Basic Management Commands}
-
-The most important \path{xm} commands are:
-\begin{quote}
-\verb_# xm list_: Lists all domains running.\\
-\verb_# xm consoles_ : Gives information about the domain consoles.\\
-\verb_# xm console_: Opens a console to a domain (e.g.\
- \verb_# xm console myVM_
-\end{quote}
-
-\subsection{\tt xm list}
-
-The output of \path{xm list} is in rows of the following format:
-\begin{center}
-{\tt name domid memory cpu state cputime console}
-\end{center}
-
-\begin{quote}
-\begin{description}
-\item[name] The descriptive name of the virtual machine.
-\item[domid] The number of the domain ID this virtual machine is running in.
-\item[memory] Memory size in megabytes.
-\item[cpu] The CPU this domain is running on.
-\item[state] Domain state consists of 5 fields:
- \begin{description}
- \item[r] running
- \item[b] blocked
- \item[p] paused
- \item[s] shutdown
- \item[c] crashed
- \end{description}
-\item[cputime] How much CPU time (in seconds) the domain has used so far.
-\item[console] TCP port accepting connections to the domain's console.
-\end{description}
-\end{quote}
-
-The \path{xm list} command also supports a long output format when the
-\path{-l} switch is used. This outputs the fulls details of the
-running domains in \xend's SXP configuration format.
-
-For example, suppose the system is running the ttylinux domain as
-described earlier. The list command should produce output somewhat
-like the following:
-\begin{verbatim}
-# xm list
-Name Id Mem(MB) CPU State Time(s) Console
-Domain-0 0 251 0 r---- 172.2
-ttylinux 5 63 0 -b--- 3.0 9605
-\end{verbatim}
-
-Here we can see the details for the ttylinux domain, as well as for
-domain 0 (which, of course, is always running). Note that the console
-port for the ttylinux domain is 9605. This can be connected to by TCP
-using a terminal program (e.g. \path{telnet} or, better,
-\path{xencons}). The simplest way to connect is to use the \path{xm console}
-command, specifying the domain name or ID. To connect to the console
-of the ttylinux domain, we could use any of the following:
-\begin{verbatim}
-# xm console ttylinux
-# xm console 5
-# xencons localhost 9605
-\end{verbatim}
-
-\section{Domain Save and Restore}
-
-The administrator of a Xen system may suspend a virtual machine's
-current state into a disk file in domain 0, allowing it to be resumed
-at a later time.
-
-The ttylinux domain described earlier can be suspended to disk using
-the command:
-\begin{verbatim}
-# xm save ttylinux ttylinux.xen
-\end{verbatim}
-
-This will stop the domain named `ttylinux' and save its current state
-into a file called \path{ttylinux.xen}.
-
-To resume execution of this domain, use the \path{xm restore} command:
-\begin{verbatim}
-# xm restore ttylinux.xen
-\end{verbatim}
-
-This will restore the state of the domain and restart it. The domain
-will carry on as before and the console may be reconnected using the
-\path{xm console} command, as above.
-
-\section{Live Migration}
-
-Live migration is used to transfer a domain between physical hosts
-whilst that domain continues to perform its usual activities --- from
-the user's perspective, the migration should be imperceptible.
-
-To perform a live migration, both hosts must be running Xen / \xend and
-the destination host must have sufficient resources (e.g. memory
-capacity) to accommodate the domain after the move. Furthermore we
-currently require both source and destination machines to be on the
-same L2 subnet.
-
-Currently, there is no support for providing automatic remote access
-to filesystems stored on local disk when a domain is migrated.
-Administrators should choose an appropriate storage solution
-(i.e. SAN, NAS, etc.) to ensure that domain filesystems are also
-available on their destination node. GNBD is a good method for
-exporting a volume from one machine to another. iSCSI can do a similar
-job, but is more complex to set up.
-
-When a domain migrates, it's MAC and IP address move with it, thus it
-is only possible to migrate VMs within the same layer-2 network and IP
-subnet. If the destination node is on a different subnet, the
-administrator would need to manually configure a suitable etherip or
-IP tunnel in the domain 0 of the remote node.
-
-A domain may be migrated using the \path{xm migrate} command. To
-live migrate a domain to another machine, we would use
-the command:
-
-\begin{verbatim}
-# xm migrate --live mydomain destination.ournetwork.com
-\end{verbatim}
-
-Without the \path{--live} flag, \xend simply stops the domain and
-copies the memory image over to the new node and restarts it. Since
-domains can have large allocations this can be quite time consuming,
-even on a Gigabit network. With the \path{--live} flag \xend attempts
-to keep the domain running while the migration is in progress,
-resulting in typical `downtimes' of just 60--300ms.
-
-For now it will be necessary to reconnect to the domain's console on
-the new machine using the \path{xm console} command. If a migrated
-domain has any open network connections then they will be preserved,
-so SSH connections do not have this limitation.
-
-\section{Managing Domain Memory}
-
-XenLinux domains have the ability to relinquish / reclaim machine
-memory at the request of the administrator or the user of the domain.
-
-\subsection{Setting memory footprints from dom0}
-
-The machine administrator can request that a domain alter its memory
-footprint using the \path{xm set-mem} command. For instance, we can
-request that our example ttylinux domain reduce its memory footprint
-to 32 megabytes.
-
-\begin{verbatim}
-# xm set-mem ttylinux 32
-\end{verbatim}
-
-We can now see the result of this in the output of \path{xm list}:
-
-\begin{verbatim}
-# xm list
-Name Id Mem(MB) CPU State Time(s) Console
-Domain-0 0 251 0 r---- 172.2
-ttylinux 5 31 0 -b--- 4.3 9605
-\end{verbatim}
-
-The domain has responded to the request by returning memory to Xen. We
-can restore the domain to its original size using the command line:
-
-\begin{verbatim}
-# xm set-mem ttylinux 64
-\end{verbatim}
-
-\subsection{Setting memory footprints from within a domain}
-
-The virtual file \path{/proc/xen/balloon} allows the owner of a
-domain to adjust their own memory footprint. Reading the file
-(e.g. \path{cat /proc/xen/balloon}) prints out the current
-memory footprint of the domain. Writing the file
-(e.g. \path{echo new\_target > /proc/xen/balloon}) requests
-that the kernel adjust the domain's memory footprint to a new value.
-
-\subsection{Setting memory limits}
-
-Xen associates a memory size limit with each domain. By default, this
-is the amount of memory the domain is originally started with,
-preventing the domain from ever growing beyond this size. To permit a
-domain to grow beyond its original allocation or to prevent a domain
-you've shrunk from reclaiming the memory it relinquished, use the
-\path{xm maxmem} command.
-
-\chapter{Domain Filesystem Storage}
-
-It is possible to directly export any Linux block device in dom0 to
-another domain, or to export filesystems / devices to virtual machines
-using standard network protocols (e.g. NBD, iSCSI, NFS, etc). This
-chapter covers some of the possibilities.
-
-
-\section{Exporting Physical Devices as VBDs}
-\label{s:exporting-physical-devices-as-vbds}
-
-One of the simplest configurations is to directly export
-individual partitions from domain 0 to other domains. To
-achieve this use the \path{phy:} specifier in your domain
-configuration file. For example a line like
-\begin{quote}
-\verb_disk = ['phy:hda3,sda1,w']_
-\end{quote}
-specifies that the partition \path{/dev/hda3} in domain 0
-should be exported read-write to the new domain as \path{/dev/sda1};
-one could equally well export it as \path{/dev/hda} or
-\path{/dev/sdb5} should one wish.
-
-In addition to local disks and partitions, it is possible to export
-any device that Linux considers to be ``a disk'' in the same manner.
-For example, if you have iSCSI disks or GNBD volumes imported into
-domain 0 you can export these to other domains using the \path{phy:}
-disk syntax. E.g.:
-\begin{quote}
-\verb_disk = ['phy:vg/lvm1,sda2,w']_
-\end{quote}
-
-
-
-\begin{center}
-\framebox{\bf Warning: Block device sharing}
-\end{center}
-\begin{quote}
-Block devices should typically only be shared between domains in a
-read-only fashion otherwise the Linux kernel's file systems will get
-very confused as the file system structure may change underneath them
-(having the same ext3 partition mounted rw twice is a sure fire way to
-cause irreparable damage)! \Xend will attempt to prevent you from
-doing this by checking that the device is not mounted read-write in
-domain 0, and hasn't already been exported read-write to another
-domain.
-If you want read-write sharing, export the directory to other domains
-via NFS from domain0 (or use a cluster file system such as GFS or
-ocfs2).
-
-\end{quote}
-
-
-\section{Using File-backed VBDs}
-
-It is also possible to use a file in Domain 0 as the primary storage
-for a virtual machine. As well as being convenient, this also has the
-advantage that the virtual block device will be {\em sparse} --- space
-will only really be allocated as parts of the file are used. So if a
-virtual machine uses only half of its disk space then the file really
-takes up half of the size allocated.
-
-For example, to create a 2GB sparse file-backed virtual block device
-(actually only consumes 1KB of disk):
-\begin{quote}
-\verb_# dd if=/dev/zero of=vm1disk bs=1k seek=2048k count=1_
-\end{quote}
-
-Make a file system in the disk file:
-\begin{quote}
-\verb_# mkfs -t ext3 vm1disk_
-\end{quote}
-
-(when the tool asks for confirmation, answer `y')
-
-Populate the file system e.g. by copying from the current root:
-\begin{quote}
-\begin{verbatim}
-# mount -o loop vm1disk /mnt
-# cp -ax /{root,dev,var,etc,usr,bin,sbin,lib} /mnt
-# mkdir /mnt/{proc,sys,home,tmp}
-\end{verbatim}
-\end{quote}
-
-Tailor the file system by editing \path{/etc/fstab},
-\path{/etc/hostname}, etc (don't forget to edit the files in the
-mounted file system, instead of your domain 0 filesystem, e.g. you
-would edit \path{/mnt/etc/fstab} instead of \path{/etc/fstab} ). For
-this example put \path{/dev/sda1} to root in fstab.
-
-Now unmount (this is important!):
-\begin{quote}
-\verb_# umount /mnt_
-\end{quote}
-
-In the configuration file set:
-\begin{quote}
-\verb_disk = ['file:/full/path/to/vm1disk,sda1,w']_
-\end{quote}
-
-As the virtual machine writes to its `disk', the sparse file will be
-filled in and consume more space up to the original 2GB.
-
-{\bf Note that file-backed VBDs may not be appropriate for backing
-I/O-intensive domains.} File-backed VBDs are known to experience
-substantial slowdowns under heavy I/O workloads, due to the I/O handling
-by the loopback block device used to support file-backed VBDs in dom0.
-Better I/O performance can be achieved by using either LVM-backed VBDs
-(Section~\ref{s:using-lvm-backed-vbds}) or physical devices as VBDs
-(Section~\ref{s:exporting-physical-devices-as-vbds}).
-
-Linux supports a maximum of eight file-backed VBDs across all domains by
-default. This limit can be statically increased by using the {\em
-max\_loop} module parameter if CONFIG\_BLK\_DEV\_LOOP is compiled as a
-module in the dom0 kernel, or by using the {\em max\_loop=n} boot option
-if CONFIG\_BLK\_DEV\_LOOP is compiled directly into the dom0 kernel.
-
-
-\section{Using LVM-backed VBDs}
-\label{s:using-lvm-backed-vbds}
-
-A particularly appealing solution is to use LVM volumes
-as backing for domain file-systems since this allows dynamic
-growing/shrinking of volumes as well as snapshot and other
-features.
-
-To initialise a partition to support LVM volumes:
-\begin{quote}
-\begin{verbatim}
-# pvcreate /dev/sda10
-\end{verbatim}
-\end{quote}
-
-Create a volume group named `vg' on the physical partition:
-\begin{quote}
-\begin{verbatim}
-# vgcreate vg /dev/sda10
-\end{verbatim}
-\end{quote}
-
-Create a logical volume of size 4GB named `myvmdisk1':
-\begin{quote}
-\begin{verbatim}
-# lvcreate -L4096M -n myvmdisk1 vg
-\end{verbatim}
-\end{quote}
-
-You should now see that you have a \path{/dev/vg/myvmdisk1}
-Make a filesystem, mount it and populate it, e.g.:
-\begin{quote}
-\begin{verbatim}
-# mkfs -t ext3 /dev/vg/myvmdisk1
-# mount /dev/vg/myvmdisk1 /mnt
-# cp -ax / /mnt
-# umount /mnt
-\end{verbatim}
-\end{quote}
-
-Now configure your VM with the following disk configuration:
-\begin{quote}
-\begin{verbatim}
- disk = [ 'phy:vg/myvmdisk1,sda1,w' ]
-\end{verbatim}
-\end{quote}
-
-LVM enables you to grow the size of logical volumes, but you'll need
-to resize the corresponding file system to make use of the new
-space. Some file systems (e.g. ext3) now support on-line resize. See
-the LVM manuals for more details.
-
-You can also use LVM for creating copy-on-write clones of LVM
-volumes (known as writable persistent snapshots in LVM
-terminology). This facility is new in Linux 2.6.8, so isn't as
-stable as one might hope. In particular, using lots of CoW LVM
-disks consumes a lot of dom0 memory, and error conditions such as
-running out of disk space are not handled well. Hopefully this
-will improve in future.
-
-To create two copy-on-write clone of the above file system you
-would use the following commands:
-
-\begin{quote}
-\begin{verbatim}
-# lvcreate -s -L1024M -n myclonedisk1 /dev/vg/myvmdisk1
-# lvcreate -s -L1024M -n myclonedisk2 /dev/vg/myvmdisk1
-\end{verbatim}
-\end{quote}
-
-Each of these can grow to have 1GB of differences from the master
-volume. You can grow the amount of space for storing the
-differences using the lvextend command, e.g.:
-\begin{quote}
-\begin{verbatim}
-# lvextend +100M /dev/vg/myclonedisk1
-\end{verbatim}
-\end{quote}
-
-Don't let the `differences volume' ever fill up otherwise LVM gets
-rather confused. It may be possible to automate the growing
-process by using \path{dmsetup wait} to spot the volume getting full
-and then issue an \path{lvextend}.
-
-In principle, it is possible to continue writing to the volume
-that has been cloned (the changes will not be visible to the
-clones), but we wouldn't recommend this: have the cloned volume
-as a `pristine' file system install that isn't mounted directly
-by any of the virtual machines.
-
-
-\section{Using NFS Root}
-
-First, populate a root filesystem in a directory on the server
-machine. This can be on a distinct physical machine, or simply
-run within a virtual machine on the same node.
-
-Now configure the NFS server to export this filesystem over the
-network by adding a line to \path{/etc/exports}, for instance:
-
-\begin{quote}
-\begin{small}
-\begin{verbatim}
-/export/vm1root 1.2.3.4/24 (rw,sync,no_root_squash)
-\end{verbatim}
-\end{small}
-\end{quote}
-
-Finally, configure the domain to use NFS root. In addition to the
-normal variables, you should make sure to set the following values in
-the domain's configuration file:
-
-\begin{quote}
-\begin{small}
-\begin{verbatim}
-root = '/dev/nfs'
-nfs_server = '2.3.4.5' # substitute IP address of server
-nfs_root = '/path/to/root' # path to root FS on the server
-\end{verbatim}
-\end{small}
-\end{quote}
-
-The domain will need network access at boot time, so either statically
-configure an IP address (Using the config variables \path{ip},
-\path{netmask}, \path{gateway}, \path{hostname}) or enable DHCP (
-\path{dhcp='dhcp'}).
-
-Note that the Linux NFS root implementation is known to have stability
-problems under high load (this is not a Xen-specific problem), so this
-configuration may not be appropriate for critical servers.
+
+%% Chapter Introduction moved to introduction.tex
+\include{src/user/introduction}
+
+%% Chapter Installation moved to installation.tex
+\include{src/user/installation}
+
+%% Chapter Starting Additional Domains moved to start_addl_dom.tex
+\include{src/user/start_addl_dom}
+
+%% Chapter Domain Management Tools moved to domain_mgmt.tex
+\include{src/user/domain_mgmt}
+
+%% Chapter Domain Filesystem Storage moved to domain_filesystem.tex
+\include{src/user/domain_filesystem}
+
\part{User Reference Documentation}
-\chapter{Control Software}
-
-The Xen control software includes the \xend node control daemon (which
-must be running), the xm command line tools, and the prototype
-xensv web interface.
-
-\section{\Xend (node control daemon)}
-\label{s:xend}
-
-The Xen Daemon (\Xend) performs system management functions related to
-virtual machines. It forms a central point of control for a machine
-and can be controlled using an HTTP-based protocol. \Xend must be
-running in order to start and manage virtual machines.
-
-\Xend must be run as root because it needs access to privileged system
-management functions. A small set of commands may be issued on the
-\xend command line:
-
-\begin{tabular}{ll}
-\verb!# xend start! & start \xend, if not already running \\
-\verb!# xend stop! & stop \xend if already running \\
-\verb!# xend restart! & restart \xend if running, otherwise start it \\
-% \verb!# xend trace_start! & start \xend, with very detailed debug logging \\
-\verb!# xend status! & indicates \xend status by its return code
-\end{tabular}
-
-A SysV init script called {\tt xend} is provided to start \xend at boot
-time. {\tt make install} installs this script in {\path{/etc/init.d}.
-To enable it, you have to make symbolic links in the appropriate
-runlevel directories or use the {\tt chkconfig} tool, where available.
-
-Once \xend is running, more sophisticated administration can be done
-using the xm tool (see Section~\ref{s:xm}) and the experimental
-Xensv web interface (see Section~\ref{s:xensv}).
-
-As \xend runs, events will be logged to \path{/var/log/xend.log} and,
-if the migration assistant daemon (\path{xfrd}) has been started,
-\path{/var/log/xfrd.log}. These may be of use for troubleshooting
-problems.
-
-\section{Xm (command line interface)}
-\label{s:xm}
-
-The xm tool is the primary tool for managing Xen from the console.
-The general format of an xm command line is:
-
-\begin{verbatim}
-# xm command [switches] [arguments] [variables]
-\end{verbatim}
-
-The available {\em switches} and {\em arguments} are dependent on the
-{\em command} chosen. The {\em variables} may be set using
-declarations of the form {\tt variable=value} and command line
-declarations override any of the values in the configuration file
-being used, including the standard variables described above and any
-custom variables (for instance, the \path{xmdefconfig} file uses a
-{\tt vmid} variable).
-
-The available commands are as follows:
-
-\begin{description}
-\item[set-mem] Request a domain to adjust its memory footprint.
-\item[create] Create a new domain.
-\item[destroy] Kill a domain immediately.
-\item[list] List running domains.
-\item[shutdown] Ask a domain to shutdown.
-\item[dmesg] Fetch the Xen (not Linux!) boot output.
-\item[consoles] Lists the available consoles.
-\item[console] Connect to the console for a domain.
-\item[help] Get help on xm commands.
-\item[save] Suspend a domain to disk.
-\item[restore] Restore a domain from disk.
-\item[pause] Pause a domain's execution.
-\item[unpause] Unpause a domain.
-\item[pincpu] Pin a domain to a CPU.
-\item[bvt] Set BVT scheduler parameters for a domain.
-\item[bvt\_ctxallow] Set the BVT context switching allowance for the system.
-\item[atropos] Set the atropos parameters for a domain.
-\item[rrobin] Set the round robin time slice for the system.
-\item[info] Get information about the Xen host.
-\item[call] Call a \xend HTTP API function directly.
-\end{description}
-
-For a detailed overview of switches, arguments and variables to each command
-try
-\begin{quote}
-\begin{verbatim}
-# xm help command
-\end{verbatim}
-\end{quote}
-
-\section{Xensv (web control interface)}
-\label{s:xensv}
-
-Xensv is the experimental web control interface for managing a Xen
-machine. It can be used to perform some (but not yet all) of the
-management tasks that can be done using the xm tool.
-
-It can be started using:
-\begin{quote}
-\verb_# xensv start_
-\end{quote}
-and stopped using:
-\begin{quote}
-\verb_# xensv stop_
-\end{quote}
-
-By default, Xensv will serve out the web interface on port 8080. This
-can be changed by editing
-\path{/usr/lib/python2.3/site-packages/xen/sv/params.py}.
-
-Once Xensv is running, the web interface can be used to create and
-manage running domains.
-
-
-
-
-\chapter{Domain Configuration}
-\label{cha:config}
-
-The following contains the syntax of the domain configuration
-files and description of how to further specify networking,
-driver domain and general scheduling behaviour.
-
-\section{Configuration Files}
-\label{s:cfiles}
-
-Xen configuration files contain the following standard variables.
-Unless otherwise stated, configuration items should be enclosed in
-quotes: see \path{/etc/xen/xmexample1} and \path{/etc/xen/xmexample2}
-for concrete examples of the syntax.
-
-\begin{description}
-\item[kernel] Path to the kernel image
-\item[ramdisk] Path to a ramdisk image (optional).
-% \item[builder] The name of the domain build function (e.g. {\tt'linux'} or
{\tt'netbsd'}.
-\item[memory] Memory size in megabytes.
-\item[cpu] CPU to run this domain on, or {\tt -1} for
- auto-allocation.
-\item[console] Port to export the domain console on (default 9600 + domain ID).
-\item[nics] Number of virtual network interfaces.
-\item[vif] List of MAC addresses (random addresses are assigned if not
- given) and bridges to use for the domain's network interfaces, e.g.
-\begin{verbatim}
-vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0',
- 'bridge=xen-br1' ]
-\end{verbatim}
- to assign a MAC address and bridge to the first interface and assign
- a different bridge to the second interface, leaving \xend to choose
- the MAC address.
-\item[disk] List of block devices to export to the domain, e.g. \\
- \verb_disk = [ 'phy:hda1,sda1,r' ]_ \\
- exports physical device \path{/dev/hda1} to the domain
- as \path{/dev/sda1} with read-only access. Exporting a disk read-write
- which is currently mounted is dangerous -- if you are \emph{certain}
- you wish to do this, you can specify \path{w!} as the mode.
-\item[dhcp] Set to {\tt 'dhcp'} if you want to use DHCP to configure
- networking.
-\item[netmask] Manually configured IP netmask.
-\item[gateway] Manually configured IP gateway.
-\item[hostname] Set the hostname for the virtual machine.
-\item[root] Specify the root device parameter on the kernel command
- line.
-\item[nfs\_server] IP address for the NFS server (if any).
-\item[nfs\_root] Path of the root filesystem on the NFS server (if any).
-\item[extra] Extra string to append to the kernel command line (if
- any)
-\item[restart] Three possible options:
- \begin{description}
- \item[always] Always restart the domain, no matter what
- its exit code is.
- \item[never] Never restart the domain.
- \item[onreboot] Restart the domain iff it requests reboot.
- \end{description}
-\end{description}
-
-For additional flexibility, it is also possible to include Python
-scripting commands in configuration files. An example of this is the
-\path{xmexample2} file, which uses Python code to handle the
-\path{vmid} variable.
-
-
-%\part{Advanced Topics}
-
-\section{Network Configuration}
-
-For many users, the default installation should work `out of the box'.
-More complicated network setups, for instance with multiple ethernet
-interfaces and/or existing bridging setups will require some
-special configuration.
-
-The purpose of this section is to describe the mechanisms provided by
-\xend to allow a flexible configuration for Xen's virtual networking.
-
-\subsection{Xen virtual network topology}
-
-Each domain network interface is connected to a virtual network
-interface in dom0 by a point to point link (effectively a `virtual
-crossover cable'). These devices are named {\tt
-vif$<$domid$>$.$<$vifid$>$} (e.g. {\tt vif1.0} for the first interface
-in domain 1, {\tt vif3.1} for the second interface in domain 3).
-
-Traffic on these virtual interfaces is handled in domain 0 using
-standard Linux mechanisms for bridging, routing, rate limiting, etc.
-Xend calls on two shell scripts to perform initial configuration of
-the network and configuration of new virtual interfaces. By default,
-these scripts configure a single bridge for all the virtual
-interfaces. Arbitrary routing / bridging configurations can be
-configured by customising the scripts, as described in the following
-section.
-
-\subsection{Xen networking scripts}
-
-Xen's virtual networking is configured by two shell scripts (by
-default \path{network} and \path{vif-bridge}). These are
-called automatically by \xend when certain events occur, with
-arguments to the scripts providing further contextual information.
-These scripts are found by default in \path{/etc/xen/scripts}. The
-names and locations of the scripts can be configured in
-\path{/etc/xen/xend-config.sxp}.
-
-\begin{description}
-
-\item[network:] This script is called whenever \xend is started or
-stopped to respectively initialise or tear down the Xen virtual
-network. In the default configuration initialisation creates the
-bridge `xen-br0' and moves eth0 onto that bridge, modifying the
-routing accordingly. When \xend exits, it deletes the Xen bridge and
-removes eth0, restoring the normal IP and routing configuration.
-
-%% In configurations where the bridge already exists, this script could
-%% be replaced with a link to \path{/bin/true} (for instance).
-
-\item[vif-bridge:] This script is called for every domain virtual
-interface and can configure firewalling rules and add the vif
-to the appropriate bridge. By default, this adds and removes
-VIFs on the default Xen bridge.
-
-\end{description}
-
-For more complex network setups (e.g. where routing is required or
-integrate with existing bridges) these scripts may be replaced with
-customised variants for your site's preferred configuration.
-
-%% There are two possible types of privileges: IO privileges and
-%% administration privileges.
-
-\section{Driver Domain Configuration}
-
-I/O privileges can be assigned to allow a domain to directly access
-PCI devices itself. This is used to support driver domains.
-
-Setting backend privileges is currently only supported in SXP format
-config files. To allow a domain to function as a backend for others,
-somewhere within the {\tt vm} element of its configuration file must
-be a {\tt backend} element of the form {\tt (backend ({\em type}))}
-where {\tt \em type} may be either {\tt netif} or {\tt blkif},
-according to the type of virtual device this domain will service.
-%% After this domain has been built, \xend will connect all new and
-%% existing {\em virtual} devices (of the appropriate type) to that
-%% backend.
-
-Note that a block backend cannot currently import virtual block
-devices from other domains, and a network backend cannot import
-virtual network devices from other domains. Thus (particularly in the
-case of block backends, which cannot import a virtual block device as
-their root filesystem), you may need to boot a backend domain from a
-ramdisk or a network device.
-
-Access to PCI devices may be configured on a per-device basis. Xen
-will assign the minimal set of hardware privileges to a domain that
-are required to control its devices. This can be configured in either
-format of configuration file:
-
-\begin{itemize}
-\item SXP Format: Include device elements of the form: \\
-\centerline{ {\tt (device (pci (bus {\em x}) (dev {\em y}) (func {\em z})))}}
\\
- inside the top-level {\tt vm} element. Each one specifies the address
- of a device this domain is allowed to access ---
- the numbers {\em x},{\em y} and {\em z} may be in either decimal or
- hexadecimal format.
-\item Flat Format: Include a list of PCI device addresses of the
- format: \\
-\centerline{{\tt pci = ['x,y,z', ...]}} \\
-where each element in the
- list is a string specifying the components of the PCI device
- address, separated by commas. The components ({\tt \em x}, {\tt \em
- y} and {\tt \em z}) of the list may be formatted as either decimal
- or hexadecimal.
-\end{itemize}
-
-%% \section{Administration Domains}
-
-%% Administration privileges allow a domain to use the `dom0
-%% operations' (so called because they are usually available only to
-%% domain 0). A privileged domain can build other domains, set scheduling
-%% parameters, etc.
-
-% Support for other administrative domains is not yet available... perhaps
-% we should plumb it in some time
-
-
-
-
-
-\section{Scheduler Configuration}
-\label{s:sched}
-
-
-Xen offers a boot time choice between multiple schedulers. To select
-a scheduler, pass the boot parameter {\em sched=sched\_name} to Xen,
-substituting the appropriate scheduler name. Details of the schedulers
-and their parameters are included below; future versions of the tools
-will provide a higher-level interface to these tools.
-
-It is expected that system administrators configure their system to
-use the scheduler most appropriate to their needs. Currently, the BVT
-scheduler is the recommended choice.
-
-\subsection{Borrowed Virtual Time}
-
-{\tt sched=bvt} (the default) \\
-
-BVT provides proportional fair shares of the CPU time. It has been
-observed to penalise domains that block frequently (e.g. I/O intensive
-domains), but this can be compensated for by using warping.
-
-\subsubsection{Global Parameters}
-
-\begin{description}
-\item[ctx\_allow]
- the context switch allowance is similar to the `quantum'
- in traditional schedulers. It is the minimum time that
- a scheduled domain will be allowed to run before being
- pre-empted.
-\end{description}
-
-\subsubsection{Per-domain parameters}
-
-\begin{description}
-\item[mcuadv]
- the MCU (Minimum Charging Unit) advance determines the
- proportional share of the CPU that a domain receives. It
- is set inversely proportionally to a domain's sharing weight.
-\item[warp]
- the amount of `virtual time' the domain is allowed to warp
- backwards
-\item[warpl]
- the warp limit is the maximum time a domain can run warped for
-\item[warpu]
- the unwarp requirement is the minimum time a domain must
- run unwarped for before it can warp again
-\end{description}
-
-\subsection{Atropos}
-
-{\tt sched=atropos} \\
-
-Atropos is a soft real time scheduler. It provides guarantees about
-absolute shares of the CPU, with a facility for sharing
-slack CPU time on a best-effort basis. It can provide timeliness
-guarantees for latency-sensitive domains.
-
-Every domain has an associated period and slice. The domain should
-receive `slice' nanoseconds every `period' nanoseconds. This allows
-the administrator to configure both the absolute share of the CPU a
-domain receives and the frequency with which it is scheduled.
-
-%% When
-%% domains unblock, their period is reduced to the value of the latency
-%% hint (the slice is scaled accordingly so that they still get the same
-%% proportion of the CPU). For each subsequent period, the slice and
-%% period times are doubled until they reach their original values.
-
-Note: don't overcommit the CPU when using Atropos (i.e. don't reserve
-more CPU than is available --- the utilisation should be kept to
-slightly less than 100\% in order to ensure predictable behaviour).
-
-\subsubsection{Per-domain parameters}
-
-\begin{description}
-\item[period] The regular time interval during which a domain is
- guaranteed to receive its allocation of CPU time.
-\item[slice]
- The length of time per period that a domain is guaranteed to run
- for (in the absence of voluntary yielding of the CPU).
-\item[latency]
- The latency hint is used to control how soon after
- waking up a domain it should be scheduled.
-\item[xtratime] This is a boolean flag that specifies whether a domain
- should be allowed a share of the system slack time.
-\end{description}
-
-\subsection{Round Robin}
-
-{\tt sched=rrobin} \\
-
-The round robin scheduler is included as a simple demonstration of
-Xen's internal scheduler API. It is not intended for production use.
-
-\subsubsection{Global Parameters}
-
-\begin{description}
-\item[rr\_slice]
- The maximum time each domain runs before the next
- scheduling decision is made.
-\end{description}
-
-
-
-
-
-
-
-
-
-
-
-
-\chapter{Build, Boot and Debug options}
-
-This chapter describes the build- and boot-time options
-which may be used to tailor your Xen system.
-
-\section{Xen Build Options}
-
-Xen provides a number of build-time options which should be
-set as environment variables or passed on make's command-line.
-
-\begin{description}
-\item[verbose=y] Enable debugging messages when Xen detects an unexpected
condition.
-Also enables console output from all domains.
-\item[debug=y]
-Enable debug assertions. Implies {\bf verbose=y}.
-(Primarily useful for tracing bugs in Xen).
-\item[debugger=y]
-Enable the in-Xen debugger. This can be used to debug
-Xen, guest OSes, and applications.
-\item[perfc=y]
-Enable performance counters for significant events
-within Xen. The counts can be reset or displayed
-on Xen's console via console control keys.
-\item[trace=y]
-Enable per-cpu trace buffers which log a range of
-events within Xen for collection by control
-software.
-\end{description}
-
-\section{Xen Boot Options}
-\label{s:xboot}
-
-These options are used to configure Xen's behaviour at runtime. They
-should be appended to Xen's command line, either manually or by
-editing \path{grub.conf}.
-
-\begin{description}
-\item [noreboot ]
- Don't reboot the machine automatically on errors. This is
- useful to catch debug output if you aren't catching console messages
- via the serial line.
-
-\item [nosmp ]
- Disable SMP support.
- This option is implied by `ignorebiostables'.
-
-\item [watchdog ]
- Enable NMI watchdog which can report certain failures.
-
-\item [noirqbalance ]
- Disable software IRQ balancing and affinity. This can be used on
- systems such as Dell 1850/2850 that have workarounds in hardware for
- IRQ-routing issues.
-
-\item [badpage=$<$page number$>$,$<$page number$>$, \ldots ]
- Specify a list of pages not to be allocated for use
- because they contain bad bytes. For example, if your
- memory tester says that byte 0x12345678 is bad, you would
- place `badpage=0x12345' on Xen's command line.
-
-\item [com1=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$
- com2=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$ ] \mbox{}\\
- Xen supports up to two 16550-compatible serial ports.
- For example: `com1=9600, 8n1, 0x408, 5' maps COM1 to a
- 9600-baud port, 8 data bits, no parity, 1 stop bit,
- I/O port base 0x408, IRQ 5.
- If some configuration options are standard (e.g., I/O base and IRQ),
- then only a prefix of the full configuration string need be
- specified. If the baud rate is pre-configured (e.g., by the
- bootloader) then you can specify `auto' in place of a numeric baud
- rate.
-
-\item [console=$<$specifier list$>$ ]
- Specify the destination for Xen console I/O.
- This is a comma-separated list of, for example:
-\begin{description}
- \item[vga] use VGA console and allow keyboard input
- \item[com1] use serial port com1
- \item[com2H] use serial port com2. Transmitted chars will
- have the MSB set. Received chars must have
- MSB set.
- \item[com2L] use serial port com2. Transmitted chars will
- have the MSB cleared. Received chars must
- have MSB cleared.
-\end{description}
- The latter two examples allow a single port to be
- shared by two subsystems (e.g. console and
- debugger). Sharing is controlled by MSB of each
- transmitted/received character.
- [NB. Default for this option is `com1,vga']
-
-\item [sync\_console ]
- Force synchronous console output. This is useful if you system fails
- unexpectedly before it has sent all available output to the
- console. In most cases Xen will automatically enter synchronous mode
- when an exceptional event occurs, but this option provides a manual
- fallback.
-
-\item [conswitch=$<$switch-char$><$auto-switch-char$>$ ]
- Specify how to switch serial-console input between
- Xen and DOM0. The required sequence is CTRL-$<$switch-char$>$
- pressed three times. Specifying the backtick character
- disables switching.
- The $<$auto-switch-char$>$ specifies whether Xen should
- auto-switch input to DOM0 when it boots --- if it is `x'
- then auto-switching is disabled. Any other value, or
- omitting the character, enables auto-switching.
- [NB. default switch-char is `a']
-
-\item [nmi=xxx ]
- Specify what to do with an NMI parity or I/O error. \\
- `nmi=fatal': Xen prints a diagnostic and then hangs. \\
- `nmi=dom0': Inform DOM0 of the NMI. \\
- `nmi=ignore': Ignore the NMI.
-
-\item [mem=xxx ]
- Set the physical RAM address limit. Any RAM appearing beyond this
- physical address in the memory map will be ignored. This parameter
- may be specified with a B, K, M or G suffix, representing bytes,
- kilobytes, megabytes and gigabytes respectively. The
- default unit, if no suffix is specified, is kilobytes.
-
-\item [dom0\_mem=xxx ]
- Set the amount of memory to be allocated to domain0. In Xen 3.x the parameter
- may be specified with a B, K, M or G suffix, representing bytes,
- kilobytes, megabytes and gigabytes respectively; if no suffix is specified,
- the parameter defaults to kilobytes. In previous versions of Xen, suffixes
- were not supported and the value is always interpreted as kilobytes.
-
-\item [tbuf\_size=xxx ]
- Set the size of the per-cpu trace buffers, in pages
- (default 1). Note that the trace buffers are only
- enabled in debug builds. Most users can ignore
- this feature completely.
-
-\item [sched=xxx ]
- Select the CPU scheduler Xen should use. The current
- possibilities are `bvt' (default), `atropos' and `rrobin'.
- For more information see Section~\ref{s:sched}.
-
-\item [apic\_verbosity=debug,verbose ]
- Print more detailed information about local APIC and IOAPIC configuration.
-
-\item [lapic ]
- Force use of local APIC even when left disabled by uniprocessor BIOS.
-
-\item [nolapic ]
- Ignore local APIC in a uniprocessor system, even if enabled by the BIOS.
-
-\item [apic=bigsmp,default,es7000,summit ]
- Specify NUMA platform. This can usually be probed automatically.
-
-\end{description}
-
-In addition, the following options may be specified on the Xen command
-line. Since domain 0 shares responsibility for booting the platform,
-Xen will automatically propagate these options to its command
-line. These options are taken from Linux's command-line syntax with
-unchanged semantics.
-
-\begin{description}
-\item [acpi=off,force,strict,ht,noirq,\ldots ]
- Modify how Xen (and domain 0) parses the BIOS ACPI tables.
-
-\item [acpi\_skip\_timer\_override ]
- Instruct Xen (and domain 0) to ignore timer-interrupt override
- instructions specified by the BIOS ACPI tables.
-
-\item [noapic ]
- Instruct Xen (and domain 0) to ignore any IOAPICs that are present in
- the system, and instead continue to use the legacy PIC.
-
-\end{description}
-
-\section{XenLinux Boot Options}
-
-In addition to the standard Linux kernel boot options, we support:
-\begin{description}
-\item[xencons=xxx ] Specify the device node to which the Xen virtual
-console driver is attached. The following options are supported:
-\begin{center}
-\begin{tabular}{l}
-`xencons=off': disable virtual console \\
-`xencons=tty': attach console to /dev/tty1 (tty0 at boot-time) \\
-`xencons=ttyS': attach console to /dev/ttyS0
-\end{tabular}
-\end{center}
-The default is ttyS for dom0 and tty for all other domains.
-\end{description}
-
-
-
-\section{Debugging}
-\label{s:keys}
-
-Xen has a set of debugging features that can be useful to try and
-figure out what's going on. Hit 'h' on the serial line (if you
-specified a baud rate on the Xen command line) or ScrollLock-h on the
-keyboard to get a list of supported commands.
-
-If you have a crash you'll likely get a crash dump containing an EIP
-(PC) which, along with an \path{objdump -d image}, can be useful in
-figuring out what's happened. Debug a Xenlinux image just as you
-would any other Linux kernel.
-
-%% We supply a handy debug terminal program which you can find in
-%% \path{/usr/local/src/xen-2.0.bk/tools/misc/miniterm/}
-%% This should be built and executed on another machine that is connected
-%% via a null modem cable. Documentation is included.
-%% Alternatively, if the Xen machine is connected to a serial-port server
-%% then we supply a dumb TCP terminal client, {\tt xencons}.
-
-
+%% Chapter Control Software moved to control_software.tex
+\include{src/user/control_software}
+
+%% Chapter Domain Configuration moved to domain_configuration.tex
+\include{src/user/domain_configuration}
+
+%% Chapter Build, Boot and Debug Options moved to build.tex
+\include{src/user/build}
\chapter{Further Support}
@@ -1875,6 +108,7 @@
%Various HOWTOs are available in \path{docs/HOWTOS} but this content is
%being integrated into this manual.
+
\section{Online References}
The official Xen web site is found at:
@@ -1884,6 +118,7 @@
This contains links to the latest versions of all on-line
documentation (including the lateset version of the FAQ).
+
\section{Mailing Lists}
@@ -1905,326 +140,18 @@
\end{description}
+
\appendix
-
-\chapter{Installing Xen / XenLinux on Debian}
-
-The Debian project provides a tool called \path{debootstrap} which
-allows a base Debian system to be installed into a filesystem without
-requiring the host system to have any Debian-specific software (such
-as \path{apt}.
-
-Here's some info how to install Debian 3.1 (Sarge) for an unprivileged
-Xen domain:
-
-\begin{enumerate}
-\item Set up Xen 2.0 and test that it's working, as described earlier in
- this manual.
-
-\item Create disk images for root-fs and swap (alternatively, you
- might create dedicated partitions, LVM logical volumes, etc. if
- that suits your setup).
-\begin{small}\begin{verbatim}
-dd if=/dev/zero of=/path/diskimage bs=1024k count=size_in_mbytes
-dd if=/dev/zero of=/path/swapimage bs=1024k count=size_in_mbytes
-\end{verbatim}\end{small}
- If you're going to use this filesystem / disk image only as a
- `template' for other vm disk images, something like 300 MB should
- be enough.. (of course it depends what kind of packages you are
- planning to install to the template)
-
-\item Create the filesystem and initialise the swap image
-\begin{small}\begin{verbatim}
-mkfs.ext3 /path/diskimage
-mkswap /path/swapimage
-\end{verbatim}\end{small}
-
-\item Mount the disk image for installation
-\begin{small}\begin{verbatim}
-mount -o loop /path/diskimage /mnt/disk
-\end{verbatim}\end{small}
-
-\item Install \path{debootstrap}
-
-Make sure you have debootstrap installed on the host. If you are
-running Debian sarge (3.1 / testing) or unstable you can install it by
-running \path{apt-get install debootstrap}. Otherwise, it can be
-downloaded from the Debian project website.
-
-\item Install Debian base to the disk image:
-\begin{small}\begin{verbatim}
-debootstrap --arch i386 sarge /mnt/disk \
- http://ftp.<countrycode>.debian.org/debian
-\end{verbatim}\end{small}
-
-You can use any other Debian http/ftp mirror you want.
-
-\item When debootstrap completes successfully, modify settings:
-\begin{small}\begin{verbatim}
-chroot /mnt/disk /bin/bash
-\end{verbatim}\end{small}
-
-Edit the following files using vi or nano and make needed changes:
-\begin{small}\begin{verbatim}
-/etc/hostname
-/etc/hosts
-/etc/resolv.conf
-/etc/network/interfaces
-/etc/networks
-\end{verbatim}\end{small}
-
-Set up access to the services, edit:
-\begin{small}\begin{verbatim}
-/etc/hosts.deny
-/etc/hosts.allow
-/etc/inetd.conf
-\end{verbatim}\end{small}
-
-Add Debian mirror to:
-\begin{small}\begin{verbatim}
-/etc/apt/sources.list
-\end{verbatim}\end{small}
-
-Create fstab like this:
-\begin{small}\begin{verbatim}
-/dev/sda1 / ext3 errors=remount-ro 0 1
-/dev/sda2 none swap sw 0 0
-proc /proc proc defaults 0 0
-\end{verbatim}\end{small}
-
-Logout
-
-\item Unmount the disk image
-\begin{small}\begin{verbatim}
-umount /mnt/disk
-\end{verbatim}\end{small}
-
-\item Create Xen 2.0 configuration file for the new domain. You can
- use the example-configurations coming with Xen as a template.
-
- Make sure you have the following set up:
-\begin{small}\begin{verbatim}
-disk = [ 'file:/path/diskimage,sda1,w', 'file:/path/swapimage,sda2,w' ]
-root = "/dev/sda1 ro"
-\end{verbatim}\end{small}
-
-\item Start the new domain
-\begin{small}\begin{verbatim}
-xm create -f domain_config_file
-\end{verbatim}\end{small}
-
-Check that the new domain is running:
-\begin{small}\begin{verbatim}
-xm list
-\end{verbatim}\end{small}
-
-\item Attach to the console of the new domain.
- You should see something like this when starting the new domain:
-
-\begin{small}\begin{verbatim}
-Started domain testdomain2, console on port 9626
-\end{verbatim}\end{small}
-
- There you can see the ID of the console: 26. You can also list
- the consoles with \path{xm consoles} (ID is the last two
- digits of the port number.)
-
- Attach to the console:
-
-\begin{small}\begin{verbatim}
-xm console 26
-\end{verbatim}\end{small}
-
- or by telnetting to the port 9626 of localhost (the xm console
- program works better).
-
-\item Log in and run base-config
-
- As a default there's no password for the root.
-
- Check that everything looks OK, and the system started without
- errors. Check that the swap is active, and the network settings are
- correct.
-
- Run \path{/usr/sbin/base-config} to set up the Debian settings.
-
- Set up the password for root using passwd.
-
-\item Done. You can exit the console by pressing \path{Ctrl + ]}
-
-\end{enumerate}
-
-If you need to create new domains, you can just copy the contents of
-the `template'-image to the new disk images, either by mounting the
-template and the new image, and using \path{cp -a} or \path{tar} or by
-simply copying the image file. Once this is done, modify the
-image-specific settings (hostname, network settings, etc).
-
-\chapter{Installing Xen / XenLinux on Redhat or Fedora Core}
-
-When using Xen / XenLinux on a standard Linux distribution there are
-a couple of things to watch out for:
-
-Note that, because domains>0 don't have any privileged access at all,
-certain commands in the default boot sequence will fail e.g. attempts
-to update the hwclock, change the console font, update the keytable
-map, start apmd (power management), or gpm (mouse cursor). Either
-ignore the errors (they should be harmless), or remove them from the
-startup scripts. Deleting the following links are a good start:
-{\path{S24pcmcia}}, {\path{S09isdn}},
-{\path{S17keytable}}, {\path{S26apmd}},
-{\path{S85gpm}}.
-
-If you want to use a single root file system that works cleanly for
-both domain 0 and unprivileged domains, a useful trick is to use
-different 'init' run levels. For example, use
-run level 3 for domain 0, and run level 4 for other domains. This
-enables different startup scripts to be run in depending on the run
-level number passed on the kernel command line.
-
-If using NFS root files systems mounted either from an
-external server or from domain0 there are a couple of other gotchas.
-The default {\path{/etc/sysconfig/iptables}} rules block NFS, so part
-way through the boot sequence things will suddenly go dead.
-
-If you're planning on having a separate NFS {\path{/usr}} partition, the
-RH9 boot scripts don't make life easy - they attempt to mount NFS file
-systems way to late in the boot process. The easiest way I found to do
-this was to have a {\path{/linuxrc}} script run ahead of
-{\path{/sbin/init}} that mounts {\path{/usr}}:
-
-\begin{quote}
-\begin{small}\begin{verbatim}
- #!/bin/bash
- /sbin/ipconfig lo 127.0.0.1
- /sbin/portmap
- /bin/mount /usr
- exec /sbin/init "$@" <>/dev/console 2>&1
-\end{verbatim}\end{small}
-\end{quote}
-
-%$ XXX SMH: font lock fix :-)
-
-The one slight complication with the above is that
-{\path{/sbin/portmap}} is dynamically linked against
-{\path{/usr/lib/libwrap.so.0}} Since this is in
-{\path{/usr}}, it won't work. This can be solved by copying the
-file (and link) below the /usr mount point, and just let the file be
-'covered' when the mount happens.
-
-In some installations, where a shared read-only {\path{/usr}} is
-being used, it may be desirable to move other large directories over
-into the read-only {\path{/usr}}. For example, you might replace
-{\path{/bin}}, {\path{/lib}} and {\path{/sbin}} with
-links into {\path{/usr/root/bin}}, {\path{/usr/root/lib}}
-and {\path{/usr/root/sbin}} respectively. This creates other
-problems for running the {\path{/linuxrc}} script, requiring
-bash, portmap, mount, ifconfig, and a handful of other shared
-libraries to be copied below the mount point --- a simple
-statically-linked C program would solve this problem.
-
-
-
-
-\chapter{Glossary of Terms}
-
-\begin{description}
-\item[Atropos] One of the CPU schedulers provided by Xen.
- Atropos provides domains with absolute shares
- of the CPU, with timeliness guarantees and a
- mechanism for sharing out `slack time'.
-
-\item[BVT] The BVT scheduler is used to give proportional
- fair shares of the CPU to domains.
-
-\item[Exokernel] A minimal piece of privileged code, similar to
- a {\bf microkernel} but providing a more
- `hardware-like' interface to the tasks it
- manages. This is similar to a paravirtualising
- VMM like {\bf Xen} but was designed as a new
- operating system structure, rather than
- specifically to run multiple conventional OSs.
-
-\item[Domain] A domain is the execution context that
- contains a running {\bf virtual machine}.
- The relationship between virtual machines
- and domains on Xen is similar to that between
- programs and processes in an operating
- system: a virtual machine is a persistent
- entity that resides on disk (somewhat like
- a program). When it is loaded for execution,
- it runs in a domain. Each domain has a
- {\bf domain ID}.
-
-\item[Domain 0] The first domain to be started on a Xen
- machine. Domain 0 is responsible for managing
- the system.
-
-\item[Domain ID] A unique identifier for a {\bf domain},
- analogous to a process ID in an operating
- system.
-
-\item[Full virtualisation] An approach to virtualisation which
- requires no modifications to the hosted
- operating system, providing the illusion of
- a complete system of real hardware devices.
-
-\item[Hypervisor] An alternative term for {\bf VMM}, used
- because it means `beyond supervisor',
- since it is responsible for managing multiple
- `supervisor' kernels.
-
-\item[Live migration] A technique for moving a running virtual
- machine to another physical host, without
- stopping it or the services running on it.
-
-\item[Microkernel] A small base of code running at the highest
- hardware privilege level. A microkernel is
- responsible for sharing CPU and memory (and
- sometimes other devices) between less
- privileged tasks running on the system.
- This is similar to a VMM, particularly a
- {\bf paravirtualising} VMM but typically
- addressing a different problem space and
- providing different kind of interface.
-
-\item[NetBSD/Xen] A port of NetBSD to the Xen architecture.
-
-\item[Paravirtualisation] An approach to virtualisation which requires
- modifications to the operating system in
- order to run in a virtual machine. Xen
- uses paravirtualisation but preserves
- binary compatibility for user space
- applications.
-
-\item[Shadow pagetables] A technique for hiding the layout of machine
- memory from a virtual machine's operating
- system. Used in some {\bf VMMs} to provide
- the illusion of contiguous physical memory,
- in Xen this is used during
- {\bf live migration}.
-
-\item[Virtual Machine] The environment in which a hosted operating
- system runs, providing the abstraction of a
- dedicated machine. A virtual machine may
- be identical to the underlying hardware (as
- in {\bf full virtualisation}, or it may
- differ, as in {\bf paravirtualisation}.
-
-\item[VMM] Virtual Machine Monitor - the software that
- allows multiple virtual machines to be
- multiplexed on a single physical machine.
-
-\item[Xen] Xen is a paravirtualising virtual machine
- monitor, developed primarily by the
- Systems Research Group at the University
- of Cambridge Computer Laboratory.
-
-\item[XenLinux] Official name for the port of the Linux kernel
- that runs on Xen.
-
-\end{description}
+%% Chapter Installing Xen / XenLinux on Debian moved to debian.tex
+\include{src/user/debian}
+
+%% Chapter Installing Xen on Red Hat moved to redhat.tex
+\include{src/user/redhat}
+
+
+%% Chapter Glossary of Terms moved to glossary.tex
+\include{src/user/glossary}
\end{document}
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/architecture.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/architecture.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,140 @@
+\chapter{Virtual Architecture}
+
+On a Xen-based system, the hypervisor itself runs in {\it ring 0}. It
+has full access to the physical memory available in the system and is
+responsible for allocating portions of it to the domains. Guest
+operating systems run in and use {\it rings 1}, {\it 2} and {\it 3} as
+they see fit. Segmentation is used to prevent the guest OS from
+accessing the portion of the address space that is reserved for Xen.
+We expect most guest operating systems will use ring 1 for their own
+operation and place applications in ring 3.
+
+In this chapter we consider the basic virtual architecture provided by
+Xen: the basic CPU state, exception and interrupt handling, and time.
+Other aspects such as memory and device access are discussed in later
+chapters.
+
+
+\section{CPU state}
+
+All privileged state must be handled by Xen. The guest OS has no
+direct access to CR3 and is not permitted to update privileged bits in
+EFLAGS. Guest OSes use \emph{hypercalls} to invoke operations in Xen;
+these are analogous to system calls but occur from ring 1 to ring 0.
+
+A list of all hypercalls is given in Appendix~\ref{a:hypercalls}.
+
+
+\section{Exceptions}
+
+A virtual IDT is provided --- a domain can submit a table of trap
+handlers to Xen via the {\tt set\_trap\_table()} hypercall. Most trap
+handlers are identical to native x86 handlers, although the page-fault
+handler is somewhat different.
+
+
+\section{Interrupts and events}
+
+Interrupts are virtualized by mapping them to \emph{events}, which are
+delivered asynchronously to the target domain using a callback
+supplied via the {\tt set\_callbacks()} hypercall. A guest OS can map
+these events onto its standard interrupt dispatch mechanisms. Xen is
+responsible for determining the target domain that will handle each
+physical interrupt source. For more details on the binding of event
+sources to events, see Chapter~\ref{c:devices}.
+
+
+\section{Time}
+
+Guest operating systems need to be aware of the passage of both real
+(or wallclock) time and their own `virtual time' (the time for which
+they have been executing). Furthermore, Xen has a notion of time which
+is used for scheduling. The following notions of time are provided:
+
+\begin{description}
+\item[Cycle counter time.]
+
+ This provides a fine-grained time reference. The cycle counter time
+ is used to accurately extrapolate the other time references. On SMP
+ machines it is currently assumed that the cycle counter time is
+ synchronized between CPUs. The current x86-based implementation
+ achieves this within inter-CPU communication latencies.
+
+\item[System time.]
+
+ This is a 64-bit counter which holds the number of nanoseconds that
+ have elapsed since system boot.
+
+\item[Wall clock time.]
+
+ This is the time of day in a Unix-style {\tt struct timeval}
+ (seconds and microseconds since 1 January 1970, adjusted by leap
+ seconds). An NTP client hosted by {\it domain 0} can keep this
+ value accurate.
+
+\item[Domain virtual time.]
+
+ This progresses at the same pace as system time, but only while a
+ domain is executing --- it stops while a domain is de-scheduled.
+ Therefore the share of the CPU that a domain receives is indicated
+ by the rate at which its virtual time increases.
+
+\end{description}
+
+
+Xen exports timestamps for system time and wall-clock time to guest
+operating systems through a shared page of memory. Xen also provides
+the cycle counter time at the instant the timestamps were calculated,
+and the CPU frequency in Hertz. This allows the guest to extrapolate
+system and wall-clock times accurately based on the current cycle
+counter time.
+
+Since all time stamps need to be updated and read \emph{atomically}
+two version numbers are also stored in the shared info page. The first
+is incremented prior to an update, while the second is only
+incremented afterwards. Thus a guest can be sure that it read a
+consistent state by checking the two version numbers are equal.
+
+Xen includes a periodic ticker which sends a timer event to the
+currently executing domain every 10ms. The Xen scheduler also sends a
+timer event whenever a domain is scheduled; this allows the guest OS
+to adjust for the time that has passed while it has been inactive. In
+addition, Xen allows each domain to request that they receive a timer
+event sent at a specified system time by using the {\tt
+ set\_timer\_op()} hypercall. Guest OSes may use this timer to
+implement timeout values when they block.
+
+
+
+%% % akw: demoting this to a section -- not sure if there is any point
+%% % though, maybe just remove it.
+
+\section{Xen CPU Scheduling}
+
+Xen offers a uniform API for CPU schedulers. It is possible to choose
+from a number of schedulers at boot and it should be easy to add more.
+The BVT, Atropos and Round Robin schedulers are part of the normal Xen
+distribution. BVT provides proportional fair shares of the CPU to the
+running domains. Atropos can be used to reserve absolute shares of
+the CPU for each domain. Round-robin is provided as an example of
+Xen's internal scheduler API.
+
+\paragraph*{Note: SMP host support}
+Xen has always supported SMP host systems. Domains are statically
+assigned to CPUs, either at creation time or when manually pinning to
+a particular CPU. The current schedulers then run locally on each CPU
+to decide which of the assigned domains should be run there. The
+user-level control software can be used to perform coarse-grain
+load-balancing between CPUs.
+
+
+%% More information on the characteristics and use of these schedulers
+%% is available in {\tt Sched-HOWTO.txt}.
+
+
+\section{Privileged operations}
+
+Xen exports an extended interface to privileged domains (viz.\ {\it
+ Domain 0}). This allows such domains to build and boot other domains
+on the server, and provides control interfaces for managing
+scheduling, memory, networking, and block devices.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/debugging.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/debugging.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,62 @@
+\chapter{Debugging}
+
+Xen provides tools for debugging both Xen and guest OSes. Currently, the
+Pervasive Debugger provides a GDB stub, which provides facilities for symbolic
+debugging of Xen itself and of OS kernels running on top of Xen. The Trace
+Buffer provides a lightweight means to log data about Xen's internal state and
+behaviour at runtime, for later analysis.
+
+\section{Pervasive Debugger}
+
+Information on using the pervasive debugger is available in pdb.txt.
+
+
+\section{Trace Buffer}
+
+The trace buffer provides a means to observe Xen's operation from domain 0.
+Trace events, inserted at key points in Xen's code, record data that can be
+read by the {\tt xentrace} tool. Recording these events has a low overhead
+and hence the trace buffer may be useful for debugging timing-sensitive
+behaviours.
+
+\subsection{Internal API}
+
+To use the trace buffer functionality from within Xen, you must {\tt \#include
+<xen/trace.h>}, which contains definitions related to the trace buffer. Trace
+events are inserted into the buffer using the {\tt TRACE\_xD} ({\tt x} = 0, 1,
+2, 3, 4 or 5) macros. These all take an event number, plus {\tt x} additional
+(32-bit) data as their arguments. For trace buffer-enabled builds of Xen these
+will insert the event ID and data into the trace buffer, along with the current
+value of the CPU cycle-counter. For builds without the trace buffer enabled,
+the macros expand to no-ops and thus can be left in place without incurring
+overheads.
+
+\subsection{Trace-enabled builds}
+
+By default, the trace buffer is enabled only in debug builds (i.e. {\tt NDEBUG}
+is not defined). It can be enabled separately by defining {\tt TRACE\_BUFFER},
+either in {\tt <xen/config.h>} or on the gcc command line.
+
+The size (in pages) of the per-CPU trace buffers can be specified using the
+{\tt tbuf\_size=n } boot parameter to Xen. If the size is set to 0, the trace
+buffers will be disabled.
+
+\subsection{Dumping trace data}
+
+When running a trace buffer build of Xen, trace data are written continuously
+into the buffer data areas, with newer data overwriting older data. This data
+can be captured using the {\tt xentrace} program in domain 0.
+
+The {\tt xentrace} tool uses {\tt /dev/mem} in domain 0 to map the trace
+buffers into its address space. It then periodically polls all the buffers for
+new data, dumping out any new records from each buffer in turn. As a result,
+for machines with multiple (logical) CPUs, the trace buffer output will not be
+in overall chronological order.
+
+The output from {\tt xentrace} can be post-processed using {\tt
+xentrace\_cpusplit} (used to split trace data out into per-cpu log files) and
+{\tt xentrace\_format} (used to pretty-print trace data). For the predefined
+trace points, there is an example format file in {\tt tools/xentrace/formats }.
+
+For more information, see the manual pages for {\tt xentrace}, {\tt
+xentrace\_format} and {\tt xentrace\_cpusplit}.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/devices.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/devices.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,178 @@
+\chapter{Devices}
+\label{c:devices}
+
+Devices such as network and disk are exported to guests using a split
+device driver. The device driver domain, which accesses the physical
+device directly also runs a \emph{backend} driver, serving requests to
+that device from guests. Each guest will use a simple \emph{frontend}
+driver, to access the backend. Communication between these domains is
+composed of two parts: First, data is placed onto a shared memory page
+between the domains. Second, an event channel between the two domains
+is used to pass notification that data is outstanding. This
+separation of notification from data transfer allows message batching,
+and results in very efficient device access.
+
+Event channels are used extensively in device virtualization; each
+domain has a number of end-points or \emph{ports} each of which may be
+bound to one of the following \emph{event sources}:
+\begin{itemize}
+ \item a physical interrupt from a real device,
+ \item a virtual interrupt (callback) from Xen, or
+ \item a signal from another domain
+\end{itemize}
+
+Events are lightweight and do not carry much information beyond the
+source of the notification. Hence when performing bulk data transfer,
+events are typically used as synchronization primitives over a shared
+memory transport. Event channels are managed via the {\tt
+ event\_channel\_op()} hypercall; for more details see
+Section~\ref{s:idc}.
+
+This chapter focuses on some individual device interfaces available to
+Xen guests.
+
+
+\section{Network I/O}
+
+Virtual network device services are provided by shared memory
+communication with a backend domain. From the point of view of other
+domains, the backend may be viewed as a virtual ethernet switch
+element with each domain having one or more virtual network interfaces
+connected to it.
+
+\subsection{Backend Packet Handling}
+
+The backend driver is responsible for a variety of actions relating to
+the transmission and reception of packets from the physical device.
+With regard to transmission, the backend performs these key actions:
+
+\begin{itemize}
+\item {\bf Validation:} To ensure that domains do not attempt to
+ generate invalid (e.g. spoofed) traffic, the backend driver may
+ validate headers ensuring that source MAC and IP addresses match the
+ interface that they have been sent from.
+
+ Validation functions can be configured using standard firewall rules
+ ({\small{\tt iptables}} in the case of Linux).
+
+\item {\bf Scheduling:} Since a number of domains can share a single
+ physical network interface, the backend must mediate access when
+ several domains each have packets queued for transmission. This
+ general scheduling function subsumes basic shaping or rate-limiting
+ schemes.
+
+\item {\bf Logging and Accounting:} The backend domain can be
+ configured with classifier rules that control how packets are
+ accounted or logged. For example, log messages might be generated
+ whenever a domain attempts to send a TCP packet containing a SYN.
+\end{itemize}
+
+On receipt of incoming packets, the backend acts as a simple
+demultiplexer: Packets are passed to the appropriate virtual interface
+after any necessary logging and accounting have been carried out.
+
+\subsection{Data Transfer}
+
+Each virtual interface uses two ``descriptor rings'', one for
+transmit, the other for receive. Each descriptor identifies a block
+of contiguous physical memory allocated to the domain.
+
+The transmit ring carries packets to transmit from the guest to the
+backend domain. The return path of the transmit ring carries messages
+indicating that the contents have been physically transmitted and the
+backend no longer requires the associated pages of memory.
+
+To receive packets, the guest places descriptors of unused pages on
+the receive ring. The backend will return received packets by
+exchanging these pages in the domain's memory with new pages
+containing the received data, and passing back descriptors regarding
+the new packets on the ring. This zero-copy approach allows the
+backend to maintain a pool of free pages to receive packets into, and
+then deliver them to appropriate domains after examining their
+headers.
+
+% Real physical addresses are used throughout, with the domain
+% performing translation from pseudo-physical addresses if that is
+% necessary.
+
+If a domain does not keep its receive ring stocked with empty buffers
+then packets destined to it may be dropped. This provides some
+defence against receive livelock problems because an overload domain
+will cease to receive further data. Similarly, on the transmit path,
+it provides the application with feedback on the rate at which packets
+are able to leave the system.
+
+Flow control on rings is achieved by including a pair of producer
+indexes on the shared ring page. Each side will maintain a private
+consumer index indicating the next outstanding message. In this
+manner, the domains cooperate to divide the ring into two message
+lists, one in each direction. Notification is decoupled from the
+immediate placement of new messages on the ring; the event channel
+will be used to generate notification when {\em either} a certain
+number of outstanding messages are queued, {\em or} a specified number
+of nanoseconds have elapsed since the oldest message was placed on the
+ring.
+
+%% Not sure if my version is any better -- here is what was here
+%% before: Synchronization between the backend domain and the guest is
+%% achieved using counters held in shared memory that is accessible to
+%% both. Each ring has associated producer and consumer indices
+%% indicating the area in the ring that holds descriptors that contain
+%% data. After receiving {\it n} packets or {\t nanoseconds} after
+%% receiving the first packet, the hypervisor sends an event to the
+%% domain.
+
+
+\section{Block I/O}
+
+All guest OS disk access goes through the virtual block device VBD
+interface. This interface allows domains access to portions of block
+storage devices visible to the the block backend device. The VBD
+interface is a split driver, similar to the network interface
+described above. A single shared memory ring is used between the
+frontend and backend drivers, across which read and write messages are
+sent.
+
+Any block device accessible to the backend domain, including
+network-based block (iSCSI, *NBD, etc), loopback and LVM/MD devices,
+can be exported as a VBD. Each VBD is mapped to a device node in the
+guest, specified in the guest's startup configuration.
+
+Old (Xen 1.2) virtual disks are not supported under Xen 2.0, since
+similar functionality can be achieved using the more complete LVM
+system, which is already in widespread use.
+
+\subsection{Data Transfer}
+
+The single ring between the guest and the block backend supports three
+messages:
+
+\begin{description}
+\item [{\small {\tt PROBE}}:] Return a list of the VBDs available to
+ this guest from the backend. The request includes a descriptor of a
+ free page into which the reply will be written by the backend.
+
+\item [{\small {\tt READ}}:] Read data from the specified block
+ device. The front end identifies the device and location to read
+ from and attaches pages for the data to be copied to (typically via
+ DMA from the device). The backend acknowledges completed read
+ requests as they finish.
+
+\item [{\small {\tt WRITE}}:] Write data to the specified block
+ device. This functions essentially as {\small {\tt READ}}, except
+ that the data moves to the device instead of from it.
+\end{description}
+
+%% um... some old text: In overview, the same style of descriptor-ring
+%% that is used for network packets is used here. Each domain has one
+%% ring that carries operation requests to the hypervisor and carries
+%% the results back again.
+
+%% Rather than copying data, the backend simply maps the domain's
+%% buffers in order to enable direct DMA to them. The act of mapping
+%% the buffers also increases the reference counts of the underlying
+%% pages, so that the unprivileged domain cannot try to return them to
+%% the hypervisor, install them as page tables, or any other unsafe
+%% behaviour.
+%%
+%% % block API here
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/further_info.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/further_info.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,49 @@
+\chapter{Further Information}
+
+If you have questions that are not answered by this manual, the
+sources of information listed below may be of interest to you. Note
+that bug reports, suggestions and contributions related to the
+software (or the documentation) should be sent to the Xen developers'
+mailing list (address below).
+
+
+\section{Other documentation}
+
+If you are mainly interested in using (rather than developing for)
+Xen, the \emph{Xen Users' Manual} is distributed in the {\tt docs/}
+directory of the Xen source distribution.
+
+% Various HOWTOs are also available in {\tt docs/HOWTOS}.
+
+
+\section{Online references}
+
+The official Xen web site is found at:
+\begin{quote}
+{\tt http://www.cl.cam.ac.uk/Research/SRG/netos/xen/}
+\end{quote}
+
+This contains links to the latest versions of all on-line
+documentation.
+
+
+\section{Mailing lists}
+
+There are currently four official Xen mailing lists:
+
+\begin{description}
+\item[xen-devel@xxxxxxxxxxxxxxxxxxx] Used for development
+ discussions and bug reports. Subscribe at: \\
+ {\small {\tt http://lists.xensource.com/xen-devel}}
+\item[xen-users@xxxxxxxxxxxxxxxxxxx] Used for installation and usage
+ discussions and requests for help. Subscribe at: \\
+ {\small {\tt http://lists.xensource.com/xen-users}}
+\item[xen-announce@xxxxxxxxxxxxxxxxxxx] Used for announcements only.
+ Subscribe at: \\
+ {\small {\tt http://lists.xensource.com/xen-announce}}
+\item[xen-changelog@xxxxxxxxxxxxxxxxxxx] Changelog feed
+ from the unstable and 2.0 trees - developer oriented. Subscribe at: \\
+ {\small {\tt http://lists.xensource.com/xen-changelog}}
+\end{description}
+
+Of these, xen-devel is the most active.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/hypercalls.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/hypercalls.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,524 @@
+
+\newcommand{\hypercall}[1]{\vspace{2mm}{\sf #1}}
+
+\chapter{Xen Hypercalls}
+\label{a:hypercalls}
+
+Hypercalls represent the procedural interface to Xen; this appendix
+categorizes and describes the current set of hypercalls.
+
+\section{Invoking Hypercalls}
+
+Hypercalls are invoked in a manner analogous to system calls in a
+conventional operating system; a software interrupt is issued which
+vectors to an entry point within Xen. On x86\_32 machines the
+instruction required is {\tt int \$82}; the (real) IDT is setup so
+that this may only be issued from within ring 1. The particular
+hypercall to be invoked is contained in {\tt EAX} --- a list
+mapping these values to symbolic hypercall names can be found
+in {\tt xen/include/public/xen.h}.
+
+On some occasions a set of hypercalls will be required to carry
+out a higher-level function; a good example is when a guest
+operating wishes to context switch to a new process which
+requires updating various privileged CPU state. As an optimization
+for these cases, there is a generic mechanism to issue a set of
+hypercalls as a batch:
+
+\begin{quote}
+\hypercall{multicall(void *call\_list, int nr\_calls)}
+
+Execute a series of hypervisor calls; {\tt nr\_calls} is the length of
+the array of {\tt multicall\_entry\_t} structures pointed to be {\tt
+call\_list}. Each entry contains the hypercall operation code followed
+by up to 7 word-sized arguments.
+\end{quote}
+
+Note that multicalls are provided purely as an optimization; there is
+no requirement to use them when first porting a guest operating
+system.
+
+
+\section{Virtual CPU Setup}
+
+At start of day, a guest operating system needs to setup the virtual
+CPU it is executing on. This includes installing vectors for the
+virtual IDT so that the guest OS can handle interrupts, page faults,
+etc. However the very first thing a guest OS must setup is a pair
+of hypervisor callbacks: these are the entry points which Xen will
+use when it wishes to notify the guest OS of an occurrence.
+
+\begin{quote}
+\hypercall{set\_callbacks(unsigned long event\_selector, unsigned long
+ event\_address, unsigned long failsafe\_selector, unsigned long
+ failsafe\_address) }
+
+Register the normal (``event'') and failsafe callbacks for
+event processing. In each case the code segment selector and
+address within that segment are provided. The selectors must
+have RPL 1; in XenLinux we simply use the kernel's CS for both
+{\tt event\_selector} and {\tt failsafe\_selector}.
+
+The value {\tt event\_address} specifies the address of the guest OSes
+event handling and dispatch routine; the {\tt failsafe\_address}
+specifies a separate entry point which is used only if a fault occurs
+when Xen attempts to use the normal callback.
+\end{quote}
+
+
+After installing the hypervisor callbacks, the guest OS can
+install a `virtual IDT' by using the following hypercall:
+
+\begin{quote}
+\hypercall{set\_trap\_table(trap\_info\_t *table)}
+
+Install one or more entries into the per-domain
+trap handler table (essentially a software version of the IDT).
+Each entry in the array pointed to by {\tt table} includes the
+exception vector number with the corresponding segment selector
+and entry point. Most guest OSes can use the same handlers on
+Xen as when running on the real hardware; an exception is the
+page fault handler (exception vector 14) where a modified
+stack-frame layout is used.
+
+
+\end{quote}
+
+
+
+\section{Scheduling and Timer}
+
+Domains are preemptively scheduled by Xen according to the
+parameters installed by domain 0 (see Section~\ref{s:dom0ops}).
+In addition, however, a domain may choose to explicitly
+control certain behavior with the following hypercall:
+
+\begin{quote}
+\hypercall{sched\_op(unsigned long op)}
+
+Request scheduling operation from hypervisor. The options are: {\it
+yield}, {\it block}, and {\it shutdown}. {\it yield} keeps the
+calling domain runnable but may cause a reschedule if other domains
+are runnable. {\it block} removes the calling domain from the run
+queue and cause is to sleeps until an event is delivered to it. {\it
+shutdown} is used to end the domain's execution; the caller can
+additionally specify whether the domain should reboot, halt or
+suspend.
+\end{quote}
+
+To aid the implementation of a process scheduler within a guest OS,
+Xen provides a virtual programmable timer:
+
+\begin{quote}
+\hypercall{set\_timer\_op(uint64\_t timeout)}
+
+Request a timer event to be sent at the specified system time (time
+in nanoseconds since system boot). The hypercall actually passes the
+64-bit timeout value as a pair of 32-bit values.
+
+\end{quote}
+
+Note that calling {\tt set\_timer\_op()} prior to {\tt sched\_op}
+allows block-with-timeout semantics.
+
+
+\section{Page Table Management}
+
+Since guest operating systems have read-only access to their page
+tables, Xen must be involved when making any changes. The following
+multi-purpose hypercall can be used to modify page-table entries,
+update the machine-to-physical mapping table, flush the TLB, install
+a new page-table base pointer, and more.
+
+\begin{quote}
+\hypercall{mmu\_update(mmu\_update\_t *req, int count, int *success\_count)}
+
+Update the page table for the domain; a set of {\tt count} updates are
+submitted for processing in a batch, with {\tt success\_count} being
+updated to report the number of successful updates.
+
+Each element of {\tt req[]} contains a pointer (address) and value;
+the least significant 2-bits of the pointer are used to distinguish
+the type of update requested as follows:
+\begin{description}
+
+\item[\it MMU\_NORMAL\_PT\_UPDATE:] update a page directory entry or
+page table entry to the associated value; Xen will check that the
+update is safe, as described in Chapter~\ref{c:memory}.
+
+\item[\it MMU\_MACHPHYS\_UPDATE:] update an entry in the
+ machine-to-physical table. The calling domain must own the machine
+ page in question (or be privileged).
+
+\item[\it MMU\_EXTENDED\_COMMAND:] perform additional MMU operations.
+The set of additional MMU operations is considerable, and includes
+updating {\tt cr3} (or just re-installing it for a TLB flush),
+flushing the cache, installing a new LDT, or pinning \& unpinning
+page-table pages (to ensure their reference count doesn't drop to zero
+which would require a revalidation of all entries).
+
+Further extended commands are used to deal with granting and
+acquiring page ownership; see Section~\ref{s:idc}.
+
+
+\end{description}
+
+More details on the precise format of all commands can be
+found in {\tt xen/include/public/xen.h}.
+
+
+\end{quote}
+
+Explicitly updating batches of page table entries is extremely
+efficient, but can require a number of alterations to the guest
+OS. Using the writable page table mode (Chapter~\ref{c:memory}) is
+recommended for new OS ports.
+
+Regardless of which page table update mode is being used, however,
+there are some occasions (notably handling a demand page fault) where
+a guest OS will wish to modify exactly one PTE rather than a
+batch. This is catered for by the following:
+
+\begin{quote}
+\hypercall{update\_va\_mapping(unsigned long page\_nr, unsigned long
+val, \\ unsigned long flags)}
+
+Update the currently installed PTE for the page {\tt page\_nr} to
+{\tt val}. As with {\tt mmu\_update()}, Xen checks the modification
+is safe before applying it. The {\tt flags} determine which kind
+of TLB flush, if any, should follow the update.
+
+\end{quote}
+
+Finally, sufficiently privileged domains may occasionally wish to manipulate
+the pages of others:
+\begin{quote}
+
+\hypercall{update\_va\_mapping\_otherdomain(unsigned long page\_nr,
+unsigned long val, unsigned long flags, uint16\_t domid)}
+
+Identical to {\tt update\_va\_mapping()} save that the pages being
+mapped must belong to the domain {\tt domid}.
+
+\end{quote}
+
+This privileged operation is currently used by backend virtual device
+drivers to safely map pages containing I/O data.
+
+
+
+\section{Segmentation Support}
+
+Xen allows guest OSes to install a custom GDT if they require it;
+this is context switched transparently whenever a domain is
+[de]scheduled. The following hypercall is effectively a
+`safe' version of {\tt lgdt}:
+
+\begin{quote}
+\hypercall{set\_gdt(unsigned long *frame\_list, int entries)}
+
+Install a global descriptor table for a domain; {\tt frame\_list} is
+an array of up to 16 machine page frames within which the GDT resides,
+with {\tt entries} being the actual number of descriptor-entry
+slots. All page frames must be mapped read-only within the guest's
+address space, and the table must be large enough to contain Xen's
+reserved entries (see {\tt xen/include/public/arch-x86\_32.h}).
+
+\end{quote}
+
+Many guest OSes will also wish to install LDTs; this is achieved by
+using {\tt mmu\_update()} with an extended command, passing the
+linear address of the LDT base along with the number of entries. No
+special safety checks are required; Xen needs to perform this task
+simply since {\tt lldt} requires CPL 0.
+
+
+Xen also allows guest operating systems to update just an
+individual segment descriptor in the GDT or LDT:
+
+\begin{quote}
+\hypercall{update\_descriptor(unsigned long ma, unsigned long word1,
+unsigned long word2)}
+
+Update the GDT/LDT entry at machine address {\tt ma}; the new
+8-byte descriptor is stored in {\tt word1} and {\tt word2}.
+Xen performs a number of checks to ensure the descriptor is
+valid.
+
+\end{quote}
+
+Guest OSes can use the above in place of context switching entire
+LDTs (or the GDT) when the number of changing descriptors is small.
+
+\section{Context Switching}
+
+When a guest OS wishes to context switch between two processes,
+it can use the page table and segmentation hypercalls described
+above to perform the the bulk of the privileged work. In addition,
+however, it will need to invoke Xen to switch the kernel (ring 1)
+stack pointer:
+
+\begin{quote}
+\hypercall{stack\_switch(unsigned long ss, unsigned long esp)}
+
+Request kernel stack switch from hypervisor; {\tt ss} is the new
+stack segment, which {\tt esp} is the new stack pointer.
+
+\end{quote}
+
+A final useful hypercall for context switching allows ``lazy''
+save and restore of floating point state:
+
+\begin{quote}
+\hypercall{fpu\_taskswitch(void)}
+
+This call instructs Xen to set the {\tt TS} bit in the {\tt cr0}
+control register; this means that the next attempt to use floating
+point will cause a trap which the guest OS can trap. Typically it will
+then save/restore the FP state, and clear the {\tt TS} bit.
+\end{quote}
+
+This is provided as an optimization only; guest OSes can also choose
+to save and restore FP state on all context switches for simplicity.
+
+
+\section{Physical Memory Management}
+
+As mentioned previously, each domain has a maximum and current
+memory allocation. The maximum allocation, set at domain creation
+time, cannot be modified. However a domain can choose to reduce
+and subsequently grow its current allocation by using the
+following call:
+
+\begin{quote}
+\hypercall{dom\_mem\_op(unsigned int op, unsigned long *extent\_list,
+ unsigned long nr\_extents, unsigned int extent\_order)}
+
+Increase or decrease current memory allocation (as determined by
+the value of {\tt op}). Each invocation provides a list of
+extents each of which is $2^s$ pages in size,
+where $s$ is the value of {\tt extent\_order}.
+
+\end{quote}
+
+In addition to simply reducing or increasing the current memory
+allocation via a `balloon driver', this call is also useful for
+obtaining contiguous regions of machine memory when required (e.g.
+for certain PCI devices, or if using superpages).
+
+
+\section{Inter-Domain Communication}
+\label{s:idc}
+
+Xen provides a simple asynchronous notification mechanism via
+\emph{event channels}. Each domain has a set of end-points (or
+\emph{ports}) which may be bound to an event source (e.g. a physical
+IRQ, a virtual IRQ, or an port in another domain). When a pair of
+end-points in two different domains are bound together, then a `send'
+operation on one will cause an event to be received by the destination
+domain.
+
+The control and use of event channels involves the following hypercall:
+
+\begin{quote}
+\hypercall{event\_channel\_op(evtchn\_op\_t *op)}
+
+Inter-domain event-channel management; {\tt op} is a discriminated
+union which allows the following 7 operations:
+
+\begin{description}
+
+\item[\it alloc\_unbound:] allocate a free (unbound) local
+ port and prepare for connection from a specified domain.
+\item[\it bind\_virq:] bind a local port to a virtual
+IRQ; any particular VIRQ can be bound to at most one port per domain.
+\item[\it bind\_pirq:] bind a local port to a physical IRQ;
+once more, a given pIRQ can be bound to at most one port per
+domain. Furthermore the calling domain must be sufficiently
+privileged.
+\item[\it bind\_interdomain:] construct an interdomain event
+channel; in general, the target domain must have previously allocated
+an unbound port for this channel, although this can be bypassed by
+privileged domains during domain setup.
+\item[\it close:] close an interdomain event channel.
+\item[\it send:] send an event to the remote end of a
+interdomain event channel.
+\item[\it status:] determine the current status of a local port.
+\end{description}
+
+For more details see
+{\tt xen/include/public/event\_channel.h}.
+
+\end{quote}
+
+Event channels are the fundamental communication primitive between
+Xen domains and seamlessly support SMP. However they provide little
+bandwidth for communication {\sl per se}, and hence are typically
+married with a piece of shared memory to produce effective and
+high-performance inter-domain communication.
+
+Safe sharing of memory pages between guest OSes is carried out by
+granting access on a per page basis to individual domains. This is
+achieved by using the {\tt grant\_table\_op()} hypercall.
+
+\begin{quote}
+\hypercall{grant\_table\_op(unsigned int cmd, void *uop, unsigned int count)}
+
+Grant or remove access to a particular page to a particular domain.
+
+\end{quote}
+
+This is not currently widely in use by guest operating systems, but
+we intend to integrate support more fully in the near future.
+
+\section{PCI Configuration}
+
+Domains with physical device access (i.e.\ driver domains) receive
+limited access to certain PCI devices (bus address space and
+interrupts). However many guest operating systems attempt to
+determine the PCI configuration by directly access the PCI BIOS,
+which cannot be allowed for safety.
+
+Instead, Xen provides the following hypercall:
+
+\begin{quote}
+\hypercall{physdev\_op(void *physdev\_op)}
+
+Perform a PCI configuration option; depending on the value
+of {\tt physdev\_op} this can be a PCI config read, a PCI config
+write, or a small number of other queries.
+
+\end{quote}
+
+
+For examples of using {\tt physdev\_op()}, see the
+Xen-specific PCI code in the linux sparse tree.
+
+\section{Administrative Operations}
+\label{s:dom0ops}
+
+A large number of control operations are available to a sufficiently
+privileged domain (typically domain 0). These allow the creation and
+management of new domains, for example. A complete list is given
+below: for more details on any or all of these, please see
+{\tt xen/include/public/dom0\_ops.h}
+
+
+\begin{quote}
+\hypercall{dom0\_op(dom0\_op\_t *op)}
+
+Administrative domain operations for domain management. The options are:
+
+\begin{description}
+\item [\it DOM0\_CREATEDOMAIN:] create a new domain
+
+\item [\it DOM0\_PAUSEDOMAIN:] remove a domain from the scheduler run
+queue.
+
+\item [\it DOM0\_UNPAUSEDOMAIN:] mark a paused domain as schedulable
+ once again.
+
+\item [\it DOM0\_DESTROYDOMAIN:] deallocate all resources associated
+with a domain
+
+\item [\it DOM0\_GETMEMLIST:] get list of pages used by the domain
+
+\item [\it DOM0\_SCHEDCTL:]
+
+\item [\it DOM0\_ADJUSTDOM:] adjust scheduling priorities for domain
+
+\item [\it DOM0\_BUILDDOMAIN:] do final guest OS setup for domain
+
+\item [\it DOM0\_GETDOMAINFO:] get statistics about the domain
+
+\item [\it DOM0\_GETPAGEFRAMEINFO:]
+
+\item [\it DOM0\_GETPAGEFRAMEINFO2:]
+
+\item [\it DOM0\_IOPL:] set I/O privilege level
+
+\item [\it DOM0\_MSR:] read or write model specific registers
+
+\item [\it DOM0\_DEBUG:] interactively invoke the debugger
+
+\item [\it DOM0\_SETTIME:] set system time
+
+\item [\it DOM0\_READCONSOLE:] read console content from hypervisor buffer ring
+
+\item [\it DOM0\_PINCPUDOMAIN:] pin domain to a particular CPU
+
+\item [\it DOM0\_GETTBUFS:] get information about the size and location of
+ the trace buffers (only on trace-buffer enabled builds)
+
+\item [\it DOM0\_PHYSINFO:] get information about the host machine
+
+\item [\it DOM0\_PCIDEV\_ACCESS:] modify PCI device access permissions
+
+\item [\it DOM0\_SCHED\_ID:] get the ID of the current Xen scheduler
+
+\item [\it DOM0\_SHADOW\_CONTROL:] switch between shadow page-table modes
+
+\item [\it DOM0\_SETDOMAININITIALMEM:] set initial memory allocation of a
domain
+
+\item [\it DOM0\_SETDOMAINMAXMEM:] set maximum memory allocation of a domain
+
+\item [\it DOM0\_SETDOMAINVMASSIST:] set domain VM assist options
+\end{description}
+\end{quote}
+
+Most of the above are best understood by looking at the code
+implementing them (in {\tt xen/common/dom0\_ops.c}) and in
+the user-space tools that use them (mostly in {\tt tools/libxc}).
+
+\section{Debugging Hypercalls}
+
+A few additional hypercalls are mainly useful for debugging:
+
+\begin{quote}
+\hypercall{console\_io(int cmd, int count, char *str)}
+
+Use Xen to interact with the console; operations are:
+
+{\it CONSOLEIO\_write}: Output count characters from buffer str.
+
+{\it CONSOLEIO\_read}: Input at most count characters into buffer str.
+\end{quote}
+
+A pair of hypercalls allows access to the underlying debug registers:
+\begin{quote}
+\hypercall{set\_debugreg(int reg, unsigned long value)}
+
+Set debug register {\tt reg} to {\tt value}
+
+\hypercall{get\_debugreg(int reg)}
+
+Return the contents of the debug register {\tt reg}
+\end{quote}
+
+And finally:
+\begin{quote}
+\hypercall{xen\_version(int cmd)}
+
+Request Xen version number.
+\end{quote}
+
+This is useful to ensure that user-space tools are in sync
+with the underlying hypervisor.
+
+\section{Deprecated Hypercalls}
+
+Xen is under constant development and refinement; as such there
+are plans to improve the way in which various pieces of functionality
+are exposed to guest OSes.
+
+\begin{quote}
+\hypercall{vm\_assist(unsigned int cmd, unsigned int type)}
+
+Toggle various memory management modes (in particular wrritable page
+tables and superpage support).
+
+\end{quote}
+
+This is likely to be replaced with mode values in the shared
+information page since this is more resilient for resumption
+after migration or checkpoint.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/memory.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/memory.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,162 @@
+\chapter{Memory}
+\label{c:memory}
+
+Xen is responsible for managing the allocation of physical memory to
+domains, and for ensuring safe use of the paging and segmentation
+hardware.
+
+
+\section{Memory Allocation}
+
+Xen resides within a small fixed portion of physical memory; it also
+reserves the top 64MB of every virtual address space. The remaining
+physical memory is available for allocation to domains at a page
+granularity. Xen tracks the ownership and use of each page, which
+allows it to enforce secure partitioning between domains.
+
+Each domain has a maximum and current physical memory allocation. A
+guest OS may run a `balloon driver' to dynamically adjust its current
+memory allocation up to its limit.
+
+
+%% XXX SMH: I use machine and physical in the next section (which is
+%% kinda required for consistency with code); wonder if this section
+%% should use same terms?
+%%
+%% Probably.
+%%
+%% Merging this and below section at some point prob makes sense.
+
+\section{Pseudo-Physical Memory}
+
+Since physical memory is allocated and freed on a page granularity,
+there is no guarantee that a domain will receive a contiguous stretch
+of physical memory. However most operating systems do not have good
+support for operating in a fragmented physical address space. To aid
+porting such operating systems to run on top of Xen, we make a
+distinction between \emph{machine memory} and \emph{pseudo-physical
+ memory}.
+
+Put simply, machine memory refers to the entire amount of memory
+installed in the machine, including that reserved by Xen, in use by
+various domains, or currently unallocated. We consider machine memory
+to comprise a set of 4K \emph{machine page frames} numbered
+consecutively starting from 0. Machine frame numbers mean the same
+within Xen or any domain.
+
+Pseudo-physical memory, on the other hand, is a per-domain
+abstraction. It allows a guest operating system to consider its memory
+allocation to consist of a contiguous range of physical page frames
+starting at physical frame 0, despite the fact that the underlying
+machine page frames may be sparsely allocated and in any order.
+
+To achieve this, Xen maintains a globally readable {\it
+ machine-to-physical} table which records the mapping from machine
+page frames to pseudo-physical ones. In addition, each domain is
+supplied with a {\it physical-to-machine} table which performs the
+inverse mapping. Clearly the machine-to-physical table has size
+proportional to the amount of RAM installed in the machine, while each
+physical-to-machine table has size proportional to the memory
+allocation of the given domain.
+
+Architecture dependent code in guest operating systems can then use
+the two tables to provide the abstraction of pseudo-physical memory.
+In general, only certain specialized parts of the operating system
+(such as page table management) needs to understand the difference
+between machine and pseudo-physical addresses.
+
+
+\section{Page Table Updates}
+
+In the default mode of operation, Xen enforces read-only access to
+page tables and requires guest operating systems to explicitly request
+any modifications. Xen validates all such requests and only applies
+updates that it deems safe. This is necessary to prevent domains from
+adding arbitrary mappings to their page tables.
+
+To aid validation, Xen associates a type and reference count with each
+memory page. A page has one of the following mutually-exclusive types
+at any point in time: page directory ({\sf PD}), page table ({\sf
+ PT}), local descriptor table ({\sf LDT}), global descriptor table
+({\sf GDT}), or writable ({\sf RW}). Note that a guest OS may always
+create readable mappings of its own memory regardless of its current
+type.
+
+%%% XXX: possibly explain more about ref count 'lifecyle' here?
+This mechanism is used to maintain the invariants required for safety;
+for example, a domain cannot have a writable mapping to any part of a
+page table as this would require the page concerned to simultaneously
+be of types {\sf PT} and {\sf RW}.
+
+
+% \section{Writable Page Tables}
+
+Xen also provides an alternative mode of operation in which guests be
+have the illusion that their page tables are directly writable. Of
+course this is not really the case, since Xen must still validate
+modifications to ensure secure partitioning. To this end, Xen traps
+any write attempt to a memory page of type {\sf PT} (i.e., that is
+currently part of a page table). If such an access occurs, Xen
+temporarily allows write access to that page while at the same time
+\emph{disconnecting} it from the page table that is currently in use.
+This allows the guest to safely make updates to the page because the
+newly-updated entries cannot be used by the MMU until Xen revalidates
+and reconnects the page. Reconnection occurs automatically in a
+number of situations: for example, when the guest modifies a different
+page-table page, when the domain is preempted, or whenever the guest
+uses Xen's explicit page-table update interfaces.
+
+Finally, Xen also supports a form of \emph{shadow page tables} in
+which the guest OS uses a independent copy of page tables which are
+unknown to the hardware (i.e.\ which are never pointed to by {\tt
+ cr3}). Instead Xen propagates changes made to the guest's tables to
+the real ones, and vice versa. This is useful for logging page writes
+(e.g.\ for live migration or checkpoint). A full version of the shadow
+page tables also allows guest OS porting with less effort.
+
+
+\section{Segment Descriptor Tables}
+
+On boot a guest is supplied with a default GDT, which does not reside
+within its own memory allocation. If the guest wishes to use other
+than the default `flat' ring-1 and ring-3 segments that this GDT
+provides, it must register a custom GDT and/or LDT with Xen, allocated
+from its own memory. Note that a number of GDT entries are reserved by
+Xen -- any custom GDT must also include sufficient space for these
+entries.
+
+For example, the following hypercall is used to specify a new GDT:
+
+\begin{quote}
+ int {\bf set\_gdt}(unsigned long *{\em frame\_list}, int {\em
+ entries})
+
+ \emph{frame\_list}: An array of up to 16 machine page frames within
+ which the GDT resides. Any frame registered as a GDT frame may only
+ be mapped read-only within the guest's address space (e.g., no
+ writable mappings, no use as a page-table page, and so on).
+
+ \emph{entries}: The number of descriptor-entry slots in the GDT.
+ Note that the table must be large enough to contain Xen's reserved
+ entries; thus we must have `{\em entries $>$
+ LAST\_RESERVED\_GDT\_ENTRY}\ '. Note also that, after registering
+ the GDT, slots \emph{FIRST\_} through
+ \emph{LAST\_RESERVED\_GDT\_ENTRY} are no longer usable by the guest
+ and may be overwritten by Xen.
+\end{quote}
+
+The LDT is updated via the generic MMU update mechanism (i.e., via the
+{\tt mmu\_update()} hypercall.
+
+\section{Start of Day}
+
+The start-of-day environment for guest operating systems is rather
+different to that provided by the underlying hardware. In particular,
+the processor is already executing in protected mode with paging
+enabled.
+
+{\it Domain 0} is created and booted by Xen itself. For all subsequent
+domains, the analogue of the boot-loader is the {\it domain builder},
+user-space software running in {\it domain 0}. The domain builder is
+responsible for building the initial page tables for a domain and
+loading its kernel image at the appropriate virtual address.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/interface/scheduling.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/interface/scheduling.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,268 @@
+\chapter{Scheduling API}
+
+The scheduling API is used by both the schedulers described above and should
+also be used by any new schedulers. It provides a generic interface and also
+implements much of the ``boilerplate'' code.
+
+Schedulers conforming to this API are described by the following
+structure:
+
+\begin{verbatim}
+struct scheduler
+{
+ char *name; /* full name for this scheduler */
+ char *opt_name; /* option name for this scheduler */
+ unsigned int sched_id; /* ID for this scheduler */
+
+ int (*init_scheduler) ();
+ int (*alloc_task) (struct task_struct *);
+ void (*add_task) (struct task_struct *);
+ void (*free_task) (struct task_struct *);
+ void (*rem_task) (struct task_struct *);
+ void (*wake_up) (struct task_struct *);
+ void (*do_block) (struct task_struct *);
+ task_slice_t (*do_schedule) (s_time_t);
+ int (*control) (struct sched_ctl_cmd *);
+ int (*adjdom) (struct task_struct *,
+ struct sched_adjdom_cmd *);
+ s32 (*reschedule) (struct task_struct *);
+ void (*dump_settings) (void);
+ void (*dump_cpu_state) (int);
+ void (*dump_runq_el) (struct task_struct *);
+};
+\end{verbatim}
+
+The only method that {\em must} be implemented is
+{\tt do\_schedule()}. However, if there is not some implementation for the
+{\tt wake\_up()} method then waking tasks will not get put on the runqueue!
+
+The fields of the above structure are described in more detail below.
+
+\subsubsection{name}
+
+The name field should point to a descriptive ASCII string.
+
+\subsubsection{opt\_name}
+
+This field is the value of the {\tt sched=} boot-time option that will select
+this scheduler.
+
+\subsubsection{sched\_id}
+
+This is an integer that uniquely identifies this scheduler. There should be a
+macro corrsponding to this scheduler ID in {\tt <xen/sched-if.h>}.
+
+\subsubsection{init\_scheduler}
+
+\paragraph*{Purpose}
+
+This is a function for performing any scheduler-specific initialisation. For
+instance, it might allocate memory for per-CPU scheduler data and initialise it
+appropriately.
+
+\paragraph*{Call environment}
+
+This function is called after the initialisation performed by the generic
+layer. The function is called exactly once, for the scheduler that has been
+selected.
+
+\paragraph*{Return values}
+
+This should return negative on failure --- this will cause an
+immediate panic and the system will fail to boot.
+
+\subsubsection{alloc\_task}
+
+\paragraph*{Purpose}
+Called when a {\tt task\_struct} is allocated by the generic scheduler
+layer. A particular scheduler implementation may use this method to
+allocate per-task data for this task. It may use the {\tt
+sched\_priv} pointer in the {\tt task\_struct} to point to this data.
+
+\paragraph*{Call environment}
+The generic layer guarantees that the {\tt sched\_priv} field will
+remain intact from the time this method is called until the task is
+deallocated (so long as the scheduler implementation does not change
+it explicitly!).
+
+\paragraph*{Return values}
+Negative on failure.
+
+\subsubsection{add\_task}
+
+\paragraph*{Purpose}
+
+Called when a task is initially added by the generic layer.
+
+\paragraph*{Call environment}
+
+The fields in the {\tt task\_struct} are now filled out and available for use.
+Schedulers should implement appropriate initialisation of any per-task private
+information in this method.
+
+\subsubsection{free\_task}
+
+\paragraph*{Purpose}
+
+Schedulers should free the space used by any associated private data
+structures.
+
+\paragraph*{Call environment}
+
+This is called when a {\tt task\_struct} is about to be deallocated.
+The generic layer will have done generic task removal operations and
+(if implemented) called the scheduler's {\tt rem\_task} method before
+this method is called.
+
+\subsubsection{rem\_task}
+
+\paragraph*{Purpose}
+
+This is called when a task is being removed from scheduling (but is
+not yet being freed).
+
+\subsubsection{wake\_up}
+
+\paragraph*{Purpose}
+
+Called when a task is woken up, this method should put the task on the runqueue
+(or do the scheduler-specific equivalent action).
+
+\paragraph*{Call environment}
+
+The task is already set to state RUNNING.
+
+\subsubsection{do\_block}
+
+\paragraph*{Purpose}
+
+This function is called when a task is blocked. This function should
+not remove the task from the runqueue.
+
+\paragraph*{Call environment}
+
+The EVENTS\_MASTER\_ENABLE\_BIT is already set and the task state changed to
+TASK\_INTERRUPTIBLE on entry to this method. A call to the {\tt
+ do\_schedule} method will be made after this method returns, in
+order to select the next task to run.
+
+\subsubsection{do\_schedule}
+
+This method must be implemented.
+
+\paragraph*{Purpose}
+
+The method is called each time a new task must be chosen for scheduling on the
+current CPU. The current time as passed as the single argument (the current
+task can be found using the {\tt current} macro).
+
+This method should select the next task to run on this CPU and set it's minimum
+time to run as well as returning the data described below.
+
+This method should also take the appropriate action if the previous
+task has blocked, e.g. removing it from the runqueue.
+
+\paragraph*{Call environment}
+
+The other fields in the {\tt task\_struct} are updated by the generic layer,
+which also performs all Xen-specific tasks and performs the actual task switch
+(unless the previous task has been chosen again).
+
+This method is called with the {\tt schedule\_lock} held for the current CPU
+and local interrupts disabled.
+
+\paragraph*{Return values}
+
+Must return a {\tt struct task\_slice} describing what task to run and how long
+for (at maximum).
+
+\subsubsection{control}
+
+\paragraph*{Purpose}
+
+This method is called for global scheduler control operations. It takes a
+pointer to a {\tt struct sched\_ctl\_cmd}, which it should either
+source data from or populate with data, depending on the value of the
+{\tt direction} field.
+
+\paragraph*{Call environment}
+
+The generic layer guarantees that when this method is called, the
+caller selected the correct scheduler ID, hence the scheduler's
+implementation does not need to sanity-check these parts of the call.
+
+\paragraph*{Return values}
+
+This function should return the value to be passed back to user space, hence it
+should either be 0 or an appropriate errno value.
+
+\subsubsection{sched\_adjdom}
+
+\paragraph*{Purpose}
+
+This method is called to adjust the scheduling parameters of a particular
+domain, or to query their current values. The function should check
+the {\tt direction} field of the {\tt sched\_adjdom\_cmd} it receives in
+order to determine which of these operations is being performed.
+
+\paragraph*{Call environment}
+
+The generic layer guarantees that the caller has specified the correct
+control interface version and scheduler ID and that the supplied {\tt
+task\_struct} will not be deallocated during the call (hence it is not
+necessary to {\tt get\_task\_struct}).
+
+\paragraph*{Return values}
+
+This function should return the value to be passed back to user space, hence it
+should either be 0 or an appropriate errno value.
+
+\subsubsection{reschedule}
+
+\paragraph*{Purpose}
+
+This method is called to determine if a reschedule is required as a result of a
+particular task.
+
+\paragraph*{Call environment}
+The generic layer will cause a reschedule if the current domain is the idle
+task or it has exceeded its minimum time slice before a reschedule. The
+generic layer guarantees that the task passed is not currently running but is
+on the runqueue.
+
+\paragraph*{Return values}
+
+Should return a mask of CPUs to cause a reschedule on.
+
+\subsubsection{dump\_settings}
+
+\paragraph*{Purpose}
+
+If implemented, this should dump any private global settings for this
+scheduler to the console.
+
+\paragraph*{Call environment}
+
+This function is called with interrupts enabled.
+
+\subsubsection{dump\_cpu\_state}
+
+\paragraph*{Purpose}
+
+This method should dump any private settings for the specified CPU.
+
+\paragraph*{Call environment}
+
+This function is called with interrupts disabled and the {\tt schedule\_lock}
+for the specified CPU held.
+
+\subsubsection{dump\_runq\_el}
+
+\paragraph*{Purpose}
+
+This method should dump any private settings for the specified task.
+
+\paragraph*{Call environment}
+
+This function is called with interrupts disabled and the {\tt schedule\_lock}
+for the task's CPU held.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/build.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/build.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,170 @@
+\chapter{Build, Boot and Debug Options}
+
+This chapter describes the build- and boot-time options which may be
+used to tailor your Xen system.
+
+
+\section{Xen Build Options}
+
+Xen provides a number of build-time options which should be set as
+environment variables or passed on make's command-line.
+
+\begin{description}
+\item[verbose=y] Enable debugging messages when Xen detects an
+ unexpected condition. Also enables console output from all domains.
+\item[debug=y] Enable debug assertions. Implies {\bf verbose=y}.
+ (Primarily useful for tracing bugs in Xen).
+\item[debugger=y] Enable the in-Xen debugger. This can be used to
+ debug Xen, guest OSes, and applications.
+\item[perfc=y] Enable performance counters for significant events
+ within Xen. The counts can be reset or displayed on Xen's console
+ via console control keys.
+\item[trace=y] Enable per-cpu trace buffers which log a range of
+ events within Xen for collection by control software.
+\end{description}
+
+
+\section{Xen Boot Options}
+\label{s:xboot}
+
+These options are used to configure Xen's behaviour at runtime. They
+should be appended to Xen's command line, either manually or by
+editing \path{grub.conf}.
+
+\begin{description}
+\item [ noreboot ] Don't reboot the machine automatically on errors.
+ This is useful to catch debug output if you aren't catching console
+ messages via the serial line.
+\item [ nosmp ] Disable SMP support. This option is implied by
+ `ignorebiostables'.
+\item [ watchdog ] Enable NMI watchdog which can report certain
+ failures.
+\item [ noirqbalance ] Disable software IRQ balancing and affinity.
+ This can be used on systems such as Dell 1850/2850 that have
+ workarounds in hardware for IRQ-routing issues.
+\item [ badpage=$<$page number$>$,$<$page number$>$, \ldots ] Specify
+ a list of pages not to be allocated for use because they contain bad
+ bytes. For example, if your memory tester says that byte 0x12345678
+ is bad, you would place `badpage=0x12345' on Xen's command line.
+\item [ com1=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$
+ com2=$<$baud$>$,DPS,$<$io\_base$>$,$<$irq$>$ ] \mbox{}\\
+ Xen supports up to two 16550-compatible serial ports. For example:
+ `com1=9600, 8n1, 0x408, 5' maps COM1 to a 9600-baud port, 8 data
+ bits, no parity, 1 stop bit, I/O port base 0x408, IRQ 5. If some
+ configuration options are standard (e.g., I/O base and IRQ), then
+ only a prefix of the full configuration string need be specified. If
+ the baud rate is pre-configured (e.g., by the bootloader) then you
+ can specify `auto' in place of a numeric baud rate.
+\item [ console=$<$specifier list$>$ ] Specify the destination for Xen
+ console I/O. This is a comma-separated list of, for example:
+ \begin{description}
+ \item[ vga ] Use VGA console and allow keyboard input.
+ \item[ com1 ] Use serial port com1.
+ \item[ com2H ] Use serial port com2. Transmitted chars will have the
+ MSB set. Received chars must have MSB set.
+ \item[ com2L] Use serial port com2. Transmitted chars will have the
+ MSB cleared. Received chars must have MSB cleared.
+ \end{description}
+ The latter two examples allow a single port to be shared by two
+ subsystems (e.g.\ console and debugger). Sharing is controlled by
+ MSB of each transmitted/received character. [NB. Default for this
+ option is `com1,vga']
+\item [ sync\_console ] Force synchronous console output. This is
+ useful if you system fails unexpectedly before it has sent all
+ available output to the console. In most cases Xen will
+ automatically enter synchronous mode when an exceptional event
+ occurs, but this option provides a manual fallback.
+\item [ conswitch=$<$switch-char$><$auto-switch-char$>$ ] Specify how
+ to switch serial-console input between Xen and DOM0. The required
+ sequence is CTRL-$<$switch-char$>$ pressed three times. Specifying
+ the backtick character disables switching. The
+ $<$auto-switch-char$>$ specifies whether Xen should auto-switch
+ input to DOM0 when it boots --- if it is `x' then auto-switching is
+ disabled. Any other value, or omitting the character, enables
+ auto-switching. [NB. Default switch-char is `a'.]
+\item [ nmi=xxx ]
+ Specify what to do with an NMI parity or I/O error. \\
+ `nmi=fatal': Xen prints a diagnostic and then hangs. \\
+ `nmi=dom0': Inform DOM0 of the NMI. \\
+ `nmi=ignore': Ignore the NMI.
+\item [ mem=xxx ] Set the physical RAM address limit. Any RAM
+ appearing beyond this physical address in the memory map will be
+ ignored. This parameter may be specified with a B, K, M or G suffix,
+ representing bytes, kilobytes, megabytes and gigabytes respectively.
+ The default unit, if no suffix is specified, is kilobytes.
+\item [ dom0\_mem=xxx ] Set the amount of memory to be allocated to
+ domain0. In Xen 3.x the parameter may be specified with a B, K, M or
+ G suffix, representing bytes, kilobytes, megabytes and gigabytes
+ respectively; if no suffix is specified, the parameter defaults to
+ kilobytes. In previous versions of Xen, suffixes were not supported
+ and the value is always interpreted as kilobytes.
+\item [ tbuf\_size=xxx ] Set the size of the per-cpu trace buffers, in
+ pages (default 1). Note that the trace buffers are only enabled in
+ debug builds. Most users can ignore this feature completely.
+\item [ sched=xxx ] Select the CPU scheduler Xen should use. The
+ current possibilities are `bvt' (default), `atropos' and `rrobin'.
+ For more information see Section~\ref{s:sched}.
+\item [ apic\_verbosity=debug,verbose ] Print more detailed
+ information about local APIC and IOAPIC configuration.
+\item [ lapic ] Force use of local APIC even when left disabled by
+ uniprocessor BIOS.
+\item [ nolapic ] Ignore local APIC in a uniprocessor system, even if
+ enabled by the BIOS.
+\item [ apic=bigsmp,default,es7000,summit ] Specify NUMA platform.
+ This can usually be probed automatically.
+\end{description}
+
+In addition, the following options may be specified on the Xen command
+line. Since domain 0 shares responsibility for booting the platform,
+Xen will automatically propagate these options to its command line.
+These options are taken from Linux's command-line syntax with
+unchanged semantics.
+
+\begin{description}
+\item [ acpi=off,force,strict,ht,noirq,\ldots ] Modify how Xen (and
+ domain 0) parses the BIOS ACPI tables.
+\item [ acpi\_skip\_timer\_override ] Instruct Xen (and domain~0) to
+ ignore timer-interrupt override instructions specified by the BIOS
+ ACPI tables.
+\item [ noapic ] Instruct Xen (and domain~0) to ignore any IOAPICs
+ that are present in the system, and instead continue to use the
+ legacy PIC.
+\end{description}
+
+
+\section{XenLinux Boot Options}
+
+In addition to the standard Linux kernel boot options, we support:
+\begin{description}
+\item[ xencons=xxx ] Specify the device node to which the Xen virtual
+ console driver is attached. The following options are supported:
+ \begin{center}
+ \begin{tabular}{l}
+ `xencons=off': disable virtual console \\
+ `xencons=tty': attach console to /dev/tty1 (tty0 at boot-time) \\
+ `xencons=ttyS': attach console to /dev/ttyS0
+ \end{tabular}
+\end{center}
+The default is ttyS for dom0 and tty for all other domains.
+\end{description}
+
+
+\section{Debugging}
+\label{s:keys}
+
+Xen has a set of debugging features that can be useful to try and
+figure out what's going on. Hit `h' on the serial line (if you
+specified a baud rate on the Xen command line) or ScrollLock-h on the
+keyboard to get a list of supported commands.
+
+If you have a crash you'll likely get a crash dump containing an EIP
+(PC) which, along with an \path{objdump -d image}, can be useful in
+figuring out what's happened. Debug a Xenlinux image just as you
+would any other Linux kernel.
+
+%% We supply a handy debug terminal program which you can find in
+%% \path{/usr/local/src/xen-2.0.bk/tools/misc/miniterm/} This should
+%% be built and executed on another machine that is connected via a
+%% null modem cable. Documentation is included. Alternatively, if the
+%% Xen machine is connected to a serial-port server then we supply a
+%% dumb TCP terminal client, {\tt xencons}.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/control_software.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/control_software.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,115 @@
+\chapter{Control Software}
+
+The Xen control software includes the \xend\ node control daemon
+(which must be running), the xm command line tools, and the prototype
+xensv web interface.
+
+\section{\Xend\ (node control daemon)}
+\label{s:xend}
+
+The Xen Daemon (\Xend) performs system management functions related to
+virtual machines. It forms a central point of control for a machine
+and can be controlled using an HTTP-based protocol. \Xend\ must be
+running in order to start and manage virtual machines.
+
+\Xend\ must be run as root because it needs access to privileged
+system management functions. A small set of commands may be issued on
+the \xend\ command line:
+
+\begin{tabular}{ll}
+ \verb!# xend start! & start \xend, if not already running \\
+ \verb!# xend stop! & stop \xend\ if already running \\
+ \verb!# xend restart! & restart \xend\ if running, otherwise start it \\
+ % \verb!# xend trace_start! & start \xend, with very detailed debug logging
\\
+ \verb!# xend status! & indicates \xend\ status by its return code
+\end{tabular}
+
+A SysV init script called {\tt xend} is provided to start \xend\ at
+boot time. {\tt make install} installs this script in
+\path{/etc/init.d}. To enable it, you have to make symbolic links in
+the appropriate runlevel directories or use the {\tt chkconfig} tool,
+where available.
+
+Once \xend\ is running, more sophisticated administration can be done
+using the xm tool (see Section~\ref{s:xm}) and the experimental Xensv
+web interface (see Section~\ref{s:xensv}).
+
+As \xend\ runs, events will be logged to \path{/var/log/xend.log} and,
+if the migration assistant daemon (\path{xfrd}) has been started,
+\path{/var/log/xfrd.log}. These may be of use for troubleshooting
+problems.
+
+\section{Xm (command line interface)}
+\label{s:xm}
+
+The xm tool is the primary tool for managing Xen from the console.
+The general format of an xm command line is:
+
+\begin{verbatim}
+# xm command [switches] [arguments] [variables]
+\end{verbatim}
+
+The available \emph{switches} and \emph{arguments} are dependent on
+the \emph{command} chosen. The \emph{variables} may be set using
+declarations of the form {\tt variable=value} and command line
+declarations override any of the values in the configuration file
+being used, including the standard variables described above and any
+custom variables (for instance, the \path{xmdefconfig} file uses a
+{\tt vmid} variable).
+
+The available commands are as follows:
+
+\begin{description}
+\item[set-mem] Request a domain to adjust its memory footprint.
+\item[create] Create a new domain.
+\item[destroy] Kill a domain immediately.
+\item[list] List running domains.
+\item[shutdown] Ask a domain to shutdown.
+\item[dmesg] Fetch the Xen (not Linux!) boot output.
+\item[consoles] Lists the available consoles.
+\item[console] Connect to the console for a domain.
+\item[help] Get help on xm commands.
+\item[save] Suspend a domain to disk.
+\item[restore] Restore a domain from disk.
+\item[pause] Pause a domain's execution.
+\item[unpause] Un-pause a domain.
+\item[pincpu] Pin a domain to a CPU.
+\item[bvt] Set BVT scheduler parameters for a domain.
+\item[bvt\_ctxallow] Set the BVT context switching allowance for the
+ system.
+\item[atropos] Set the atropos parameters for a domain.
+\item[rrobin] Set the round robin time slice for the system.
+\item[info] Get information about the Xen host.
+\item[call] Call a \xend\ HTTP API function directly.
+\end{description}
+
+For a detailed overview of switches, arguments and variables to each
+command try
+\begin{quote}
+\begin{verbatim}
+# xm help command
+\end{verbatim}
+\end{quote}
+
+\section{Xensv (web control interface)}
+\label{s:xensv}
+
+Xensv is the experimental web control interface for managing a Xen
+machine. It can be used to perform some (but not yet all) of the
+management tasks that can be done using the xm tool.
+
+It can be started using:
+\begin{quote}
+ \verb_# xensv start_
+\end{quote}
+and stopped using:
+\begin{quote}
+ \verb_# xensv stop_
+\end{quote}
+
+By default, Xensv will serve out the web interface on port 8080. This
+can be changed by editing
+\path{/usr/lib/python2.3/site-packages/xen/sv/params.py}.
+
+Once Xensv is running, the web interface can be used to create and
+manage running domains.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/debian.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/debian.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,154 @@
+\chapter{Installing Xen / XenLinux on Debian}
+
+The Debian project provides a tool called \path{debootstrap} which
+allows a base Debian system to be installed into a filesystem without
+requiring the host system to have any Debian-specific software (such
+as \path{apt}).
+
+Here's some info how to install Debian 3.1 (Sarge) for an unprivileged
+Xen domain:
+
+\begin{enumerate}
+
+\item Set up Xen and test that it's working, as described earlier in
+ this manual.
+
+\item Create disk images for rootfs and swap. Alternatively, you might
+ create dedicated partitions, LVM logical volumes, etc.\ if that
+ suits your setup.
+\begin{verbatim}
+dd if=/dev/zero of=/path/diskimage bs=1024k count=size_in_mbytes
+dd if=/dev/zero of=/path/swapimage bs=1024k count=size_in_mbytes
+\end{verbatim}
+
+ If you're going to use this filesystem / disk image only as a
+ `template' for other vm disk images, something like 300 MB should be
+ enough. (of course it depends what kind of packages you are planning
+ to install to the template)
+
+\item Create the filesystem and initialise the swap image
+\begin{verbatim}
+mkfs.ext3 /path/diskimage
+mkswap /path/swapimage
+\end{verbatim}
+
+\item Mount the disk image for installation
+\begin{verbatim}
+mount -o loop /path/diskimage /mnt/disk
+\end{verbatim}
+
+\item Install \path{debootstrap}. Make sure you have debootstrap
+ installed on the host. If you are running Debian Sarge (3.1 /
+ testing) or unstable you can install it by running \path{apt-get
+ install debootstrap}. Otherwise, it can be downloaded from the
+ Debian project website.
+
+\item Install Debian base to the disk image:
+\begin{verbatim}
+debootstrap --arch i386 sarge /mnt/disk \
+ http://ftp.<countrycode>.debian.org/debian
+\end{verbatim}
+
+ You can use any other Debian http/ftp mirror you want.
+
+\item When debootstrap completes successfully, modify settings:
+\begin{verbatim}
+chroot /mnt/disk /bin/bash
+\end{verbatim}
+
+Edit the following files using vi or nano and make needed changes:
+\begin{verbatim}
+/etc/hostname
+/etc/hosts
+/etc/resolv.conf
+/etc/network/interfaces
+/etc/networks
+\end{verbatim}
+
+Set up access to the services, edit:
+\begin{verbatim}
+/etc/hosts.deny
+/etc/hosts.allow
+/etc/inetd.conf
+\end{verbatim}
+
+Add Debian mirror to:
+\begin{verbatim}
+/etc/apt/sources.list
+\end{verbatim}
+
+Create fstab like this:
+\begin{verbatim}
+/dev/sda1 / ext3 errors=remount-ro 0 1
+/dev/sda2 none swap sw 0 0
+proc /proc proc defaults 0 0
+\end{verbatim}
+
+Logout
+
+\item Unmount the disk image
+\begin{verbatim}
+umount /mnt/disk
+\end{verbatim}
+
+\item Create Xen 2.0 configuration file for the new domain. You can
+ use the example-configurations coming with Xen as a template.
+
+ Make sure you have the following set up:
+\begin{verbatim}
+disk = [ 'file:/path/diskimage,sda1,w', 'file:/path/swapimage,sda2,w' ]
+root = "/dev/sda1 ro"
+\end{verbatim}
+
+\item Start the new domain
+\begin{verbatim}
+xm create -f domain_config_file
+\end{verbatim}
+
+Check that the new domain is running:
+\begin{verbatim}
+xm list
+\end{verbatim}
+
+\item Attach to the console of the new domain. You should see
+ something like this when starting the new domain:
+
+\begin{verbatim}
+Started domain testdomain2, console on port 9626
+\end{verbatim}
+
+ There you can see the ID of the console: 26. You can also list the
+ consoles with \path{xm consoles} (ID is the last two digits of the
+ port number.)
+
+ Attach to the console:
+
+\begin{verbatim}
+xm console 26
+\end{verbatim}
+
+ or by telnetting to the port 9626 of localhost (the xm console
+ program works better).
+
+\item Log in and run base-config
+
+ As a default there's no password for the root.
+
+ Check that everything looks OK, and the system started without
+ errors. Check that the swap is active, and the network settings are
+ correct.
+
+ Run \path{/usr/sbin/base-config} to set up the Debian settings.
+
+ Set up the password for root using passwd.
+
+\item Done. You can exit the console by pressing {\path{Ctrl + ]}}
+
+\end{enumerate}
+
+
+If you need to create new domains, you can just copy the contents of
+the `template'-image to the new disk images, either by mounting the
+template and the new image, and using \path{cp -a} or \path{tar} or by
+simply copying the image file. Once this is done, modify the
+image-specific settings (hostname, network settings, etc).
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/domain_configuration.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/domain_configuration.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,281 @@
+\chapter{Domain Configuration}
+\label{cha:config}
+
+The following contains the syntax of the domain configuration files
+and description of how to further specify networking, driver domain
+and general scheduling behavior.
+
+
+\section{Configuration Files}
+\label{s:cfiles}
+
+Xen configuration files contain the following standard variables.
+Unless otherwise stated, configuration items should be enclosed in
+quotes: see \path{/etc/xen/xmexample1} and \path{/etc/xen/xmexample2}
+for concrete examples of the syntax.
+
+\begin{description}
+\item[kernel] Path to the kernel image.
+\item[ramdisk] Path to a ramdisk image (optional).
+ % \item[builder] The name of the domain build function (e.g.
+ % {\tt'linux'} or {\tt'netbsd'}.
+\item[memory] Memory size in megabytes.
+\item[cpu] CPU to run this domain on, or {\tt -1} for auto-allocation.
+\item[console] Port to export the domain console on (default 9600 +
+ domain ID).
+\item[nics] Number of virtual network interfaces.
+\item[vif] List of MAC addresses (random addresses are assigned if not
+ given) and bridges to use for the domain's network interfaces, e.g.\
+\begin{verbatim}
+vif = [ 'mac=aa:00:00:00:00:11, bridge=xen-br0',
+ 'bridge=xen-br1' ]
+\end{verbatim}
+ to assign a MAC address and bridge to the first interface and assign
+ a different bridge to the second interface, leaving \xend\ to choose
+ the MAC address.
+\item[disk] List of block devices to export to the domain, e.g.\ \\
+ \verb_disk = [ 'phy:hda1,sda1,r' ]_ \\
+ exports physical device \path{/dev/hda1} to the domain as
+ \path{/dev/sda1} with read-only access. Exporting a disk read-write
+ which is currently mounted is dangerous -- if you are \emph{certain}
+ you wish to do this, you can specify \path{w!} as the mode.
+\item[dhcp] Set to {\tt `dhcp'} if you want to use DHCP to configure
+ networking.
+\item[netmask] Manually configured IP netmask.
+\item[gateway] Manually configured IP gateway.
+\item[hostname] Set the hostname for the virtual machine.
+\item[root] Specify the root device parameter on the kernel command
+ line.
+\item[nfs\_server] IP address for the NFS server (if any).
+\item[nfs\_root] Path of the root filesystem on the NFS server (if
+ any).
+\item[extra] Extra string to append to the kernel command line (if
+ any)
+\item[restart] Three possible options:
+ \begin{description}
+ \item[always] Always restart the domain, no matter what its exit
+ code is.
+ \item[never] Never restart the domain.
+ \item[onreboot] Restart the domain iff it requests reboot.
+ \end{description}
+\end{description}
+
+For additional flexibility, it is also possible to include Python
+scripting commands in configuration files. An example of this is the
+\path{xmexample2} file, which uses Python code to handle the
+\path{vmid} variable.
+
+
+%\part{Advanced Topics}
+
+
+\section{Network Configuration}
+
+For many users, the default installation should work ``out of the
+box''. More complicated network setups, for instance with multiple
+Ethernet interfaces and/or existing bridging setups will require some
+special configuration.
+
+The purpose of this section is to describe the mechanisms provided by
+\xend\ to allow a flexible configuration for Xen's virtual networking.
+
+\subsection{Xen virtual network topology}
+
+Each domain network interface is connected to a virtual network
+interface in dom0 by a point to point link (effectively a ``virtual
+crossover cable''). These devices are named {\tt
+ vif$<$domid$>$.$<$vifid$>$} (e.g.\ {\tt vif1.0} for the first
+interface in domain~1, {\tt vif3.1} for the second interface in
+domain~3).
+
+Traffic on these virtual interfaces is handled in domain~0 using
+standard Linux mechanisms for bridging, routing, rate limiting, etc.
+Xend calls on two shell scripts to perform initial configuration of
+the network and configuration of new virtual interfaces. By default,
+these scripts configure a single bridge for all the virtual
+interfaces. Arbitrary routing / bridging configurations can be
+configured by customizing the scripts, as described in the following
+section.
+
+\subsection{Xen networking scripts}
+
+Xen's virtual networking is configured by two shell scripts (by
+default \path{network} and \path{vif-bridge}). These are called
+automatically by \xend\ when certain events occur, with arguments to
+the scripts providing further contextual information. These scripts
+are found by default in \path{/etc/xen/scripts}. The names and
+locations of the scripts can be configured in
+\path{/etc/xen/xend-config.sxp}.
+
+\begin{description}
+\item[network:] This script is called whenever \xend\ is started or
+ stopped to respectively initialize or tear down the Xen virtual
+ network. In the default configuration initialization creates the
+ bridge `xen-br0' and moves eth0 onto that bridge, modifying the
+ routing accordingly. When \xend\ exits, it deletes the Xen bridge
+ and removes eth0, restoring the normal IP and routing configuration.
+
+ %% In configurations where the bridge already exists, this script
+ %% could be replaced with a link to \path{/bin/true} (for instance).
+
+\item[vif-bridge:] This script is called for every domain virtual
+ interface and can configure firewalling rules and add the vif to the
+ appropriate bridge. By default, this adds and removes VIFs on the
+ default Xen bridge.
+\end{description}
+
+For more complex network setups (e.g.\ where routing is required or
+integrate with existing bridges) these scripts may be replaced with
+customized variants for your site's preferred configuration.
+
+%% There are two possible types of privileges: IO privileges and
+%% administration privileges.
+
+
+\section{Driver Domain Configuration}
+
+I/O privileges can be assigned to allow a domain to directly access
+PCI devices itself. This is used to support driver domains.
+
+Setting back-end privileges is currently only supported in SXP format
+config files. To allow a domain to function as a back-end for others,
+somewhere within the {\tt vm} element of its configuration file must
+be a {\tt back-end} element of the form {\tt (back-end ({\em type}))}
+where {\tt \em type} may be either {\tt netif} or {\tt blkif},
+according to the type of virtual device this domain will service.
+%% After this domain has been built, \xend will connect all new and
+%% existing {\em virtual} devices (of the appropriate type) to that
+%% back-end.
+
+Note that a block back-end cannot currently import virtual block
+devices from other domains, and a network back-end cannot import
+virtual network devices from other domains. Thus (particularly in the
+case of block back-ends, which cannot import a virtual block device as
+their root filesystem), you may need to boot a back-end domain from a
+ramdisk or a network device.
+
+Access to PCI devices may be configured on a per-device basis. Xen
+will assign the minimal set of hardware privileges to a domain that
+are required to control its devices. This can be configured in either
+format of configuration file:
+
+\begin{itemize}
+\item SXP Format: Include device elements of the form: \\
+ \centerline{ {\tt (device (pci (bus {\em x}) (dev {\em y}) (func {\em
z})))}} \\
+ inside the top-level {\tt vm} element. Each one specifies the
+ address of a device this domain is allowed to access --- the numbers
+ \emph{x},\emph{y} and \emph{z} may be in either decimal or
+ hexadecimal format.
+\item Flat Format: Include a list of PCI device addresses of the
+ format: \\
+ \centerline{{\tt pci = ['x,y,z', \ldots]}} \\
+ where each element in the list is a string specifying the components
+ of the PCI device address, separated by commas. The components
+ ({\tt \em x}, {\tt \em y} and {\tt \em z}) of the list may be
+ formatted as either decimal or hexadecimal.
+\end{itemize}
+
+%% \section{Administration Domains}
+
+%% Administration privileges allow a domain to use the `dom0
+%% operations' (so called because they are usually available only to
+%% domain 0). A privileged domain can build other domains, set
+%% scheduling parameters, etc.
+
+% Support for other administrative domains is not yet available...
+% perhaps we should plumb it in some time
+
+
+\section{Scheduler Configuration}
+\label{s:sched}
+
+Xen offers a boot time choice between multiple schedulers. To select
+a scheduler, pass the boot parameter \emph{sched=sched\_name} to Xen,
+substituting the appropriate scheduler name. Details of the
+schedulers and their parameters are included below; future versions of
+the tools will provide a higher-level interface to these tools.
+
+It is expected that system administrators configure their system to
+use the scheduler most appropriate to their needs. Currently, the BVT
+scheduler is the recommended choice.
+
+\subsection{Borrowed Virtual Time}
+
+{\tt sched=bvt} (the default) \\
+
+BVT provides proportional fair shares of the CPU time. It has been
+observed to penalize domains that block frequently (e.g.\ I/O
+intensive domains), but this can be compensated for by using warping.
+
+\subsubsection{Global Parameters}
+
+\begin{description}
+\item[ctx\_allow] The context switch allowance is similar to the
+ ``quantum'' in traditional schedulers. It is the minimum time that
+ a scheduled domain will be allowed to run before being preempted.
+\end{description}
+
+\subsubsection{Per-domain parameters}
+
+\begin{description}
+\item[mcuadv] The MCU (Minimum Charging Unit) advance determines the
+ proportional share of the CPU that a domain receives. It is set
+ inversely proportionally to a domain's sharing weight.
+\item[warp] The amount of ``virtual time'' the domain is allowed to
+ warp backwards.
+\item[warpl] The warp limit is the maximum time a domain can run
+ warped for.
+\item[warpu] The unwarp requirement is the minimum time a domain must
+ run unwarped for before it can warp again.
+\end{description}
+
+\subsection{Atropos}
+
+{\tt sched=atropos} \\
+
+Atropos is a soft real time scheduler. It provides guarantees about
+absolute shares of the CPU, with a facility for sharing slack CPU time
+on a best-effort basis. It can provide timeliness guarantees for
+latency-sensitive domains.
+
+Every domain has an associated period and slice. The domain should
+receive `slice' nanoseconds every `period' nanoseconds. This allows
+the administrator to configure both the absolute share of the CPU a
+domain receives and the frequency with which it is scheduled.
+
+%% When domains unblock, their period is reduced to the value of the
+%% latency hint (the slice is scaled accordingly so that they still
+%% get the same proportion of the CPU). For each subsequent period,
+%% the slice and period times are doubled until they reach their
+%% original values.
+
+Note: don't over-commit the CPU when using Atropos (i.e.\ don't reserve
+more CPU than is available --- the utilization should be kept to
+slightly less than 100\% in order to ensure predictable behavior).
+
+\subsubsection{Per-domain parameters}
+
+\begin{description}
+\item[period] The regular time interval during which a domain is
+ guaranteed to receive its allocation of CPU time.
+\item[slice] The length of time per period that a domain is guaranteed
+ to run for (in the absence of voluntary yielding of the CPU).
+\item[latency] The latency hint is used to control how soon after
+ waking up a domain it should be scheduled.
+\item[xtratime] This is a boolean flag that specifies whether a domain
+ should be allowed a share of the system slack time.
+\end{description}
+
+\subsection{Round Robin}
+
+{\tt sched=rrobin} \\
+
+The round robin scheduler is included as a simple demonstration of
+Xen's internal scheduler API. It is not intended for production use.
+
+\subsubsection{Global Parameters}
+
+\begin{description}
+\item[rr\_slice] The maximum time each domain runs before the next
+ scheduling decision is made.
+\end{description}
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/domain_filesystem.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/domain_filesystem.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,243 @@
+\chapter{Domain Filesystem Storage}
+
+It is possible to directly export any Linux block device in dom0 to
+another domain, or to export filesystems / devices to virtual machines
+using standard network protocols (e.g.\ NBD, iSCSI, NFS, etc.). This
+chapter covers some of the possibilities.
+
+
+\section{Exporting Physical Devices as VBDs}
+\label{s:exporting-physical-devices-as-vbds}
+
+One of the simplest configurations is to directly export individual
+partitions from domain~0 to other domains. To achieve this use the
+\path{phy:} specifier in your domain configuration file. For example a
+line like
+\begin{quote}
+ \verb_disk = ['phy:hda3,sda1,w']_
+\end{quote}
+specifies that the partition \path{/dev/hda3} in domain~0 should be
+exported read-write to the new domain as \path{/dev/sda1}; one could
+equally well export it as \path{/dev/hda} or \path{/dev/sdb5} should
+one wish.
+
+In addition to local disks and partitions, it is possible to export
+any device that Linux considers to be ``a disk'' in the same manner.
+For example, if you have iSCSI disks or GNBD volumes imported into
+domain~0 you can export these to other domains using the \path{phy:}
+disk syntax. E.g.:
+\begin{quote}
+ \verb_disk = ['phy:vg/lvm1,sda2,w']_
+\end{quote}
+
+\begin{center}
+ \framebox{\bf Warning: Block device sharing}
+\end{center}
+\begin{quote}
+ Block devices should typically only be shared between domains in a
+ read-only fashion otherwise the Linux kernel's file systems will get
+ very confused as the file system structure may change underneath
+ them (having the same ext3 partition mounted \path{rw} twice is a
+ sure fire way to cause irreparable damage)! \Xend\ will attempt to
+ prevent you from doing this by checking that the device is not
+ mounted read-write in domain~0, and hasn't already been exported
+ read-write to another domain. If you want read-write sharing,
+ export the directory to other domains via NFS from domain~0 (or use
+ a cluster file system such as GFS or ocfs2).
+\end{quote}
+
+
+\section{Using File-backed VBDs}
+
+It is also possible to use a file in Domain~0 as the primary storage
+for a virtual machine. As well as being convenient, this also has the
+advantage that the virtual block device will be \emph{sparse} ---
+space will only really be allocated as parts of the file are used. So
+if a virtual machine uses only half of its disk space then the file
+really takes up half of the size allocated.
+
+For example, to create a 2GB sparse file-backed virtual block device
+(actually only consumes 1KB of disk):
+\begin{quote}
+ \verb_# dd if=/dev/zero of=vm1disk bs=1k seek=2048k count=1_
+\end{quote}
+
+Make a file system in the disk file:
+\begin{quote}
+ \verb_# mkfs -t ext3 vm1disk_
+\end{quote}
+
+(when the tool asks for confirmation, answer `y')
+
+Populate the file system e.g.\ by copying from the current root:
+\begin{quote}
+\begin{verbatim}
+# mount -o loop vm1disk /mnt
+# cp -ax /{root,dev,var,etc,usr,bin,sbin,lib} /mnt
+# mkdir /mnt/{proc,sys,home,tmp}
+\end{verbatim}
+\end{quote}
+
+Tailor the file system by editing \path{/etc/fstab},
+\path{/etc/hostname}, etc.\ Don't forget to edit the files in the
+mounted file system, instead of your domain~0 filesystem, e.g.\ you
+would edit \path{/mnt/etc/fstab} instead of \path{/etc/fstab}. For
+this example put \path{/dev/sda1} to root in fstab.
+
+Now unmount (this is important!):
+\begin{quote}
+ \verb_# umount /mnt_
+\end{quote}
+
+In the configuration file set:
+\begin{quote}
+ \verb_disk = ['file:/full/path/to/vm1disk,sda1,w']_
+\end{quote}
+
+As the virtual machine writes to its `disk', the sparse file will be
+filled in and consume more space up to the original 2GB.
+
+{\bf Note that file-backed VBDs may not be appropriate for backing
+ I/O-intensive domains.} File-backed VBDs are known to experience
+substantial slowdowns under heavy I/O workloads, due to the I/O
+handling by the loopback block device used to support file-backed VBDs
+in dom0. Better I/O performance can be achieved by using either
+LVM-backed VBDs (Section~\ref{s:using-lvm-backed-vbds}) or physical
+devices as VBDs (Section~\ref{s:exporting-physical-devices-as-vbds}).
+
+Linux supports a maximum of eight file-backed VBDs across all domains
+by default. This limit can be statically increased by using the
+\emph{max\_loop} module parameter if CONFIG\_BLK\_DEV\_LOOP is
+compiled as a module in the dom0 kernel, or by using the
+\emph{max\_loop=n} boot option if CONFIG\_BLK\_DEV\_LOOP is compiled
+directly into the dom0 kernel.
+
+
+\section{Using LVM-backed VBDs}
+\label{s:using-lvm-backed-vbds}
+
+A particularly appealing solution is to use LVM volumes as backing for
+domain file-systems since this allows dynamic growing/shrinking of
+volumes as well as snapshot and other features.
+
+To initialize a partition to support LVM volumes:
+\begin{quote}
+\begin{verbatim}
+# pvcreate /dev/sda10
+\end{verbatim}
+\end{quote}
+
+Create a volume group named `vg' on the physical partition:
+\begin{quote}
+\begin{verbatim}
+# vgcreate vg /dev/sda10
+\end{verbatim}
+\end{quote}
+
+Create a logical volume of size 4GB named `myvmdisk1':
+\begin{quote}
+\begin{verbatim}
+# lvcreate -L4096M -n myvmdisk1 vg
+\end{verbatim}
+\end{quote}
+
+You should now see that you have a \path{/dev/vg/myvmdisk1} Make a
+filesystem, mount it and populate it, e.g.:
+\begin{quote}
+\begin{verbatim}
+# mkfs -t ext3 /dev/vg/myvmdisk1
+# mount /dev/vg/myvmdisk1 /mnt
+# cp -ax / /mnt
+# umount /mnt
+\end{verbatim}
+\end{quote}
+
+Now configure your VM with the following disk configuration:
+\begin{quote}
+\begin{verbatim}
+ disk = [ 'phy:vg/myvmdisk1,sda1,w' ]
+\end{verbatim}
+\end{quote}
+
+LVM enables you to grow the size of logical volumes, but you'll need
+to resize the corresponding file system to make use of the new space.
+Some file systems (e.g.\ ext3) now support online resize. See the LVM
+manuals for more details.
+
+You can also use LVM for creating copy-on-write (CoW) clones of LVM
+volumes (known as writable persistent snapshots in LVM terminology).
+This facility is new in Linux 2.6.8, so isn't as stable as one might
+hope. In particular, using lots of CoW LVM disks consumes a lot of
+dom0 memory, and error conditions such as running out of disk space
+are not handled well. Hopefully this will improve in future.
+
+To create two copy-on-write clone of the above file system you would
+use the following commands:
+
+\begin{quote}
+\begin{verbatim}
+# lvcreate -s -L1024M -n myclonedisk1 /dev/vg/myvmdisk1
+# lvcreate -s -L1024M -n myclonedisk2 /dev/vg/myvmdisk1
+\end{verbatim}
+\end{quote}
+
+Each of these can grow to have 1GB of differences from the master
+volume. You can grow the amount of space for storing the differences
+using the lvextend command, e.g.:
+\begin{quote}
+\begin{verbatim}
+# lvextend +100M /dev/vg/myclonedisk1
+\end{verbatim}
+\end{quote}
+
+Don't let the `differences volume' ever fill up otherwise LVM gets
+rather confused. It may be possible to automate the growing process by
+using \path{dmsetup wait} to spot the volume getting full and then
+issue an \path{lvextend}.
+
+In principle, it is possible to continue writing to the volume that
+has been cloned (the changes will not be visible to the clones), but
+we wouldn't recommend this: have the cloned volume as a `pristine'
+file system install that isn't mounted directly by any of the virtual
+machines.
+
+
+\section{Using NFS Root}
+
+First, populate a root filesystem in a directory on the server
+machine. This can be on a distinct physical machine, or simply run
+within a virtual machine on the same node.
+
+Now configure the NFS server to export this filesystem over the
+network by adding a line to \path{/etc/exports}, for instance:
+
+\begin{quote}
+ \begin{small}
+\begin{verbatim}
+/export/vm1root 1.2.3.4/24 (rw,sync,no_root_squash)
+\end{verbatim}
+ \end{small}
+\end{quote}
+
+Finally, configure the domain to use NFS root. In addition to the
+normal variables, you should make sure to set the following values in
+the domain's configuration file:
+
+\begin{quote}
+ \begin{small}
+\begin{verbatim}
+root = '/dev/nfs'
+nfs_server = '2.3.4.5' # substitute IP address of server
+nfs_root = '/path/to/root' # path to root FS on the server
+\end{verbatim}
+ \end{small}
+\end{quote}
+
+The domain will need network access at boot time, so either statically
+configure an IP address using the config variables \path{ip},
+\path{netmask}, \path{gateway}, \path{hostname}; or enable DHCP
+(\path{dhcp='dhcp'}).
+
+Note that the Linux NFS root implementation is known to have stability
+problems under high load (this is not a Xen-specific problem), so this
+configuration may not be appropriate for critical servers.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/domain_mgmt.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/domain_mgmt.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,203 @@
+\chapter{Domain Management Tools}
+
+The previous chapter described a simple example of how to configure
+and start a domain. This chapter summarises the tools available to
+manage running domains.
+
+
+\section{Command-line Management}
+
+Command line management tasks are also performed using the \path{xm}
+tool. For online help for the commands available, type:
+\begin{quote}
+ \verb_# xm help_
+\end{quote}
+
+You can also type \path{xm help $<$command$>$} for more information on
+a given command.
+
+\subsection{Basic Management Commands}
+
+The most important \path{xm} commands are:
+\begin{quote}
+ \verb_# xm list_: Lists all domains running.\\
+ \verb_# xm consoles_: Gives information about the domain consoles.\\
+ \verb_# xm console_: Opens a console to a domain (e.g.\
+ \verb_# xm console myVM_)
+\end{quote}
+
+\subsection{\tt xm list}
+
+The output of \path{xm list} is in rows of the following format:
+\begin{center} {\tt name domid memory cpu state cputime console}
+\end{center}
+
+\begin{quote}
+ \begin{description}
+ \item[name] The descriptive name of the virtual machine.
+ \item[domid] The number of the domain ID this virtual machine is
+ running in.
+ \item[memory] Memory size in megabytes.
+ \item[cpu] The CPU this domain is running on.
+ \item[state] Domain state consists of 5 fields:
+ \begin{description}
+ \item[r] running
+ \item[b] blocked
+ \item[p] paused
+ \item[s] shutdown
+ \item[c] crashed
+ \end{description}
+ \item[cputime] How much CPU time (in seconds) the domain has used so
+ far.
+ \item[console] TCP port accepting connections to the domain's
+ console.
+ \end{description}
+\end{quote}
+
+The \path{xm list} command also supports a long output format when the
+\path{-l} switch is used. This outputs the fulls details of the
+running domains in \xend's SXP configuration format.
+
+For example, suppose the system is running the ttylinux domain as
+described earlier. The list command should produce output somewhat
+like the following:
+\begin{verbatim}
+# xm list
+Name Id Mem(MB) CPU State Time(s) Console
+Domain-0 0 251 0 r---- 172.2
+ttylinux 5 63 0 -b--- 3.0 9605
+\end{verbatim}
+
+Here we can see the details for the ttylinux domain, as well as for
+domain~0 (which, of course, is always running). Note that the console
+port for the ttylinux domain is 9605. This can be connected to by TCP
+using a terminal program (e.g. \path{telnet} or, better,
+\path{xencons}). The simplest way to connect is to use the
+\path{xm~console} command, specifying the domain name or ID. To
+connect to the console of the ttylinux domain, we could use any of the
+following:
+\begin{verbatim}
+# xm console ttylinux
+# xm console 5
+# xencons localhost 9605
+\end{verbatim}
+
+\section{Domain Save and Restore}
+
+The administrator of a Xen system may suspend a virtual machine's
+current state into a disk file in domain~0, allowing it to be resumed
+at a later time.
+
+The ttylinux domain described earlier can be suspended to disk using
+the command:
+\begin{verbatim}
+# xm save ttylinux ttylinux.xen
+\end{verbatim}
+
+This will stop the domain named `ttylinux' and save its current state
+into a file called \path{ttylinux.xen}.
+
+To resume execution of this domain, use the \path{xm restore} command:
+\begin{verbatim}
+# xm restore ttylinux.xen
+\end{verbatim}
+
+This will restore the state of the domain and restart it. The domain
+will carry on as before and the console may be reconnected using the
+\path{xm console} command, as above.
+
+\section{Live Migration}
+
+Live migration is used to transfer a domain between physical hosts
+whilst that domain continues to perform its usual activities --- from
+the user's perspective, the migration should be imperceptible.
+
+To perform a live migration, both hosts must be running Xen / \xend\
+and the destination host must have sufficient resources (e.g.\ memory
+capacity) to accommodate the domain after the move. Furthermore we
+currently require both source and destination machines to be on the
+same L2 subnet.
+
+Currently, there is no support for providing automatic remote access
+to filesystems stored on local disk when a domain is migrated.
+Administrators should choose an appropriate storage solution (i.e.\
+SAN, NAS, etc.) to ensure that domain filesystems are also available
+on their destination node. GNBD is a good method for exporting a
+volume from one machine to another. iSCSI can do a similar job, but is
+more complex to set up.
+
+When a domain migrates, it's MAC and IP address move with it, thus it
+is only possible to migrate VMs within the same layer-2 network and IP
+subnet. If the destination node is on a different subnet, the
+administrator would need to manually configure a suitable etherip or
+IP tunnel in the domain~0 of the remote node.
+
+A domain may be migrated using the \path{xm migrate} command. To live
+migrate a domain to another machine, we would use the command:
+
+\begin{verbatim}
+# xm migrate --live mydomain destination.ournetwork.com
+\end{verbatim}
+
+Without the \path{--live} flag, \xend\ simply stops the domain and
+copies the memory image over to the new node and restarts it. Since
+domains can have large allocations this can be quite time consuming,
+even on a Gigabit network. With the \path{--live} flag \xend\ attempts
+to keep the domain running while the migration is in progress,
+resulting in typical `downtimes' of just 60--300ms.
+
+For now it will be necessary to reconnect to the domain's console on
+the new machine using the \path{xm console} command. If a migrated
+domain has any open network connections then they will be preserved,
+so SSH connections do not have this limitation.
+
+
+\section{Managing Domain Memory}
+
+XenLinux domains have the ability to relinquish / reclaim machine
+memory at the request of the administrator or the user of the domain.
+
+\subsection{Setting memory footprints from dom0}
+
+The machine administrator can request that a domain alter its memory
+footprint using the \path{xm set-mem} command. For instance, we can
+request that our example ttylinux domain reduce its memory footprint
+to 32 megabytes.
+
+\begin{verbatim}
+# xm set-mem ttylinux 32
+\end{verbatim}
+
+We can now see the result of this in the output of \path{xm list}:
+
+\begin{verbatim}
+# xm list
+Name Id Mem(MB) CPU State Time(s) Console
+Domain-0 0 251 0 r---- 172.2
+ttylinux 5 31 0 -b--- 4.3 9605
+\end{verbatim}
+
+The domain has responded to the request by returning memory to Xen. We
+can restore the domain to its original size using the command line:
+
+\begin{verbatim}
+# xm set-mem ttylinux 64
+\end{verbatim}
+
+\subsection{Setting memory footprints from within a domain}
+
+The virtual file \path{/proc/xen/balloon} allows the owner of a domain
+to adjust their own memory footprint. Reading the file (e.g.\
+\path{cat /proc/xen/balloon}) prints out the current memory footprint
+of the domain. Writing the file (e.g.\ \path{echo new\_target >
+ /proc/xen/balloon}) requests that the kernel adjust the domain's
+memory footprint to a new value.
+
+\subsection{Setting memory limits}
+
+Xen associates a memory size limit with each domain. By default, this
+is the amount of memory the domain is originally started with,
+preventing the domain from ever growing beyond this size. To permit a
+domain to grow beyond its original allocation or to prevent a domain
+you've shrunk from reclaiming the memory it relinquished, use the
+\path{xm maxmem} command.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/glossary.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/glossary.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,79 @@
+\chapter{Glossary of Terms}
+
+\begin{description}
+
+\item[Atropos] One of the CPU schedulers provided by Xen. Atropos
+ provides domains with absolute shares of the CPU, with timeliness
+ guarantees and a mechanism for sharing out `slack time'.
+
+\item[BVT] The BVT scheduler is used to give proportional fair shares
+ of the CPU to domains.
+
+\item[Exokernel] A minimal piece of privileged code, similar to a {\bf
+ microkernel} but providing a more `hardware-like' interface to the
+ tasks it manages. This is similar to a paravirtualising VMM like
+ {\bf Xen} but was designed as a new operating system structure,
+ rather than specifically to run multiple conventional OSs.
+
+\item[Domain] A domain is the execution context that contains a
+ running {\bf virtual machine}. The relationship between virtual
+ machines and domains on Xen is similar to that between programs and
+ processes in an operating system: a virtual machine is a persistent
+ entity that resides on disk (somewhat like a program). When it is
+ loaded for execution, it runs in a domain. Each domain has a {\bf
+ domain ID}.
+
+\item[Domain 0] The first domain to be started on a Xen machine.
+ Domain 0 is responsible for managing the system.
+
+\item[Domain ID] A unique identifier for a {\bf domain}, analogous to
+ a process ID in an operating system.
+
+\item[Full virtualisation] An approach to virtualisation which
+ requires no modifications to the hosted operating system, providing
+ the illusion of a complete system of real hardware devices.
+
+\item[Hypervisor] An alternative term for {\bf VMM}, used because it
+ means `beyond supervisor', since it is responsible for managing
+ multiple `supervisor' kernels.
+
+\item[Live migration] A technique for moving a running virtual machine
+ to another physical host, without stopping it or the services
+ running on it.
+
+\item[Microkernel] A small base of code running at the highest
+ hardware privilege level. A microkernel is responsible for sharing
+ CPU and memory (and sometimes other devices) between less privileged
+ tasks running on the system. This is similar to a VMM, particularly
+ a {\bf paravirtualising} VMM but typically addressing a different
+ problem space and providing different kind of interface.
+
+\item[NetBSD/Xen] A port of NetBSD to the Xen architecture.
+
+\item[Paravirtualisation] An approach to virtualisation which requires
+ modifications to the operating system in order to run in a virtual
+ machine. Xen uses paravirtualisation but preserves binary
+ compatibility for user space applications.
+
+\item[Shadow pagetables] A technique for hiding the layout of machine
+ memory from a virtual machine's operating system. Used in some {\bf
+ VMMs} to provide the illusion of contiguous physical memory, in
+ Xen this is used during {\bf live migration}.
+
+\item[Virtual Machine] The environment in which a hosted operating
+ system runs, providing the abstraction of a dedicated machine. A
+ virtual machine may be identical to the underlying hardware (as in
+ {\bf full virtualisation}, or it may differ, as in {\bf
+ paravirtualisation}).
+
+\item[VMM] Virtual Machine Monitor - the software that allows multiple
+ virtual machines to be multiplexed on a single physical machine.
+
+\item[Xen] Xen is a paravirtualising virtual machine monitor,
+ developed primarily by the Systems Research Group at the University
+ of Cambridge Computer Laboratory.
+
+\item[XenLinux] Official name for the port of the Linux kernel that
+ runs on Xen.
+
+\end{description}
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/installation.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/installation.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,394 @@
+\chapter{Installation}
+
+The Xen distribution includes three main components: Xen itself, ports
+of Linux 2.4 and 2.6 and NetBSD to run on Xen, and the userspace
+tools required to manage a Xen-based system. This chapter describes
+how to install the Xen~2.0 distribution from source. Alternatively,
+there may be pre-built packages available as part of your operating
+system distribution.
+
+
+\section{Prerequisites}
+\label{sec:prerequisites}
+
+The following is a full list of prerequisites. Items marked `$\dag$'
+are required by the \xend\ control tools, and hence required if you
+want to run more than one virtual machine; items marked `$*$' are only
+required if you wish to build from source.
+\begin{itemize}
+\item A working Linux distribution using the GRUB bootloader and
+ running on a P6-class (or newer) CPU.
+\item [$\dag$] The \path{iproute2} package.
+\item [$\dag$] The Linux bridge-utils\footnote{Available from {\tt
+ http://bridge.sourceforge.net}} (e.g., \path{/sbin/brctl})
+\item [$\dag$] An installation of Twisted~v1.3 or
+ above\footnote{Available from {\tt http://www.twistedmatrix.com}}.
+ There may be a binary package available for your distribution;
+ alternatively it can be installed by running `{\sl make
+ install-twisted}' in the root of the Xen source tree.
+\item [$*$] Build tools (gcc v3.2.x or v3.3.x, binutils, GNU make).
+\item [$*$] Development installation of libcurl (e.g., libcurl-devel)
+\item [$*$] Development installation of zlib (e.g., zlib-dev).
+\item [$*$] Development installation of Python v2.2 or later (e.g.,
+ python-dev).
+\item [$*$] \LaTeX\ and transfig are required to build the
+ documentation.
+\end{itemize}
+
+Once you have satisfied the relevant prerequisites, you can now
+install either a binary or source distribution of Xen.
+
+
+\section{Installing from Binary Tarball}
+
+Pre-built tarballs are available for download from the Xen download
+page
+\begin{quote} {\tt http://xen.sf.net}
+\end{quote}
+
+Once you've downloaded the tarball, simply unpack and install:
+\begin{verbatim}
+# tar zxvf xen-2.0-install.tgz
+# cd xen-2.0-install
+# sh ./install.sh
+\end{verbatim}
+
+Once you've installed the binaries you need to configure your system
+as described in Section~\ref{s:configure}.
+
+
+\section{Installing from Source}
+
+This section describes how to obtain, build, and install Xen from
+source.
+
+\subsection{Obtaining the Source}
+
+The Xen source tree is available as either a compressed source tar
+ball or as a clone of our master BitKeeper repository.
+
+\begin{description}
+\item[Obtaining the Source Tarball]\mbox{} \\
+ Stable versions (and daily snapshots) of the Xen source tree are
+ available as compressed tarballs from the Xen download page
+ \begin{quote} {\tt http://xen.sf.net}
+ \end{quote}
+
+\item[Using BitKeeper]\mbox{} \\
+ If you wish to install Xen from a clone of our latest BitKeeper
+ repository then you will need to install the BitKeeper tools.
+ Download instructions for BitKeeper can be obtained by filling out
+ the form at:
+ \begin{quote} {\tt http://www.bitmover.com/cgi-bin/download.cgi}
+\end{quote}
+The public master BK repository for the 2.0 release lives at:
+\begin{quote} {\tt bk://xen.bkbits.net/xen-2.0.bk}
+\end{quote}
+You can use BitKeeper to download it and keep it updated with the
+latest features and fixes.
+
+Change to the directory in which you want to put the source code, then
+run:
+\begin{verbatim}
+# bk clone bk://xen.bkbits.net/xen-2.0.bk
+\end{verbatim}
+
+Under your current directory, a new directory named \path{xen-2.0.bk}
+has been created, which contains all the source code for Xen, the OS
+ports, and the control tools. You can update your repository with the
+latest changes at any time by running:
+\begin{verbatim}
+# cd xen-2.0.bk # to change into the local repository
+# bk pull # to update the repository
+\end{verbatim}
+\end{description}
+
+% \section{The distribution}
+%
+% The Xen source code repository is structured as follows:
+%
+% \begin{description}
+% \item[\path{tools/}] Xen node controller daemon (Xend), command line
+% tools, control libraries
+% \item[\path{xen/}] The Xen VMM.
+% \item[\path{linux-*-xen-sparse/}] Xen support for Linux.
+% \item[\path{linux-*-patches/}] Experimental patches for Linux.
+% \item[\path{netbsd-*-xen-sparse/}] Xen support for NetBSD.
+% \item[\path{docs/}] Various documentation files for users and
+% developers.
+% \item[\path{extras/}] Bonus extras.
+% \end{description}
+
+\subsection{Building from Source}
+
+The top-level Xen Makefile includes a target `world' that will do the
+following:
+
+\begin{itemize}
+\item Build Xen.
+\item Build the control tools, including \xend.
+\item Download (if necessary) and unpack the Linux 2.6 source code,
+ and patch it for use with Xen.
+\item Build a Linux kernel to use in domain 0 and a smaller
+ unprivileged kernel, which can optionally be used for unprivileged
+ virtual machines.
+\end{itemize}
+
+After the build has completed you should have a top-level directory
+called \path{dist/} in which all resulting targets will be placed; of
+particular interest are the two kernels XenLinux kernel images, one
+with a `-xen0' extension which contains hardware device drivers and
+drivers for Xen's virtual devices, and one with a `-xenU' extension
+that just contains the virtual ones. These are found in
+\path{dist/install/boot/} along with the image for Xen itself and the
+configuration files used during the build.
+
+The NetBSD port can be built using:
+\begin{quote}
+\begin{verbatim}
+# make netbsd20
+\end{verbatim}
+\end{quote}
+NetBSD port is built using a snapshot of the netbsd-2-0 cvs branch.
+The snapshot is downloaded as part of the build process, if it is not
+yet present in the \path{NETBSD\_SRC\_PATH} search path. The build
+process also downloads a toolchain which includes all the tools
+necessary to build the NetBSD kernel under Linux.
+
+To customize further the set of kernels built you need to edit the
+top-level Makefile. Look for the line:
+
+\begin{quote}
+\begin{verbatim}
+KERNELS ?= mk.linux-2.6-xen0 mk.linux-2.6-xenU
+\end{verbatim}
+\end{quote}
+
+You can edit this line to include any set of operating system kernels
+which have configurations in the top-level \path{buildconfigs/}
+directory, for example \path{mk.linux-2.4-xenU} to build a Linux 2.4
+kernel containing only virtual device drivers.
+
+%% Inspect the Makefile if you want to see what goes on during a
+%% build. Building Xen and the tools is straightforward, but XenLinux
+%% is more complicated. The makefile needs a `pristine' Linux kernel
+%% tree to which it will then add the Xen architecture files. You can
+%% tell the makefile the location of the appropriate Linux compressed
+%% tar file by
+%% setting the LINUX\_SRC environment variable, e.g. \\
+%% \verb!# LINUX_SRC=/tmp/linux-2.6.11.tar.bz2 make world! \\ or by
+%% placing the tar file somewhere in the search path of {\tt
+%% LINUX\_SRC\_PATH} which defaults to `{\tt .:..}'. If the
+%% makefile can't find a suitable kernel tar file it attempts to
+%% download it from kernel.org (this won't work if you're behind a
+%% firewall).
+
+%% After untaring the pristine kernel tree, the makefile uses the {\tt
+%% mkbuildtree} script to add the Xen patches to the kernel.
+
+
+%% The procedure is similar to build the Linux 2.4 port: \\
+%% \verb!# LINUX_SRC=/path/to/linux2.4/source make linux24!
+
+
+%% \framebox{\parbox{5in}{
+%% {\bf Distro specific:} \\
+%% {\it Gentoo} --- if not using udev (most installations,
+%% currently), you'll need to enable devfs and devfs mount at boot
+%% time in the xen0 config. }}
+
+\subsection{Custom XenLinux Builds}
+
+% If you have an SMP machine you may wish to give the {\tt '-j4'}
+% argument to make to get a parallel build.
+
+If you wish to build a customized XenLinux kernel (e.g. to support
+additional devices or enable distribution-required features), you can
+use the standard Linux configuration mechanisms, specifying that the
+architecture being built for is \path{xen}, e.g:
+\begin{quote}
+\begin{verbatim}
+# cd linux-2.6.11-xen0
+# make ARCH=xen xconfig
+# cd ..
+# make
+\end{verbatim}
+\end{quote}
+
+You can also copy an existing Linux configuration (\path{.config})
+into \path{linux-2.6.11-xen0} and execute:
+\begin{quote}
+\begin{verbatim}
+# make ARCH=xen oldconfig
+\end{verbatim}
+\end{quote}
+
+You may be prompted with some Xen-specific options; we advise
+accepting the defaults for these options.
+
+Note that the only difference between the two types of Linux kernel
+that are built is the configuration file used for each. The `U'
+suffixed (unprivileged) versions don't contain any of the physical
+hardware device drivers, leading to a 30\% reduction in size; hence
+you may prefer these for your non-privileged domains. The `0'
+suffixed privileged versions can be used to boot the system, as well
+as in driver domains and unprivileged domains.
+
+\subsection{Installing the Binaries}
+
+The files produced by the build process are stored under the
+\path{dist/install/} directory. To install them in their default
+locations, do:
+\begin{quote}
+\begin{verbatim}
+# make install
+\end{verbatim}
+\end{quote}
+
+Alternatively, users with special installation requirements may wish
+to install them manually by copying the files to their appropriate
+destinations.
+
+%% Files in \path{install/boot/} include:
+%% \begin{itemize}
+%% \item \path{install/boot/xen-2.0.gz} Link to the Xen 'kernel'
+%% \item \path{install/boot/vmlinuz-2.6-xen0} Link to domain 0
+%% XenLinux kernel
+%% \item \path{install/boot/vmlinuz-2.6-xenU} Link to unprivileged
+%% XenLinux kernel
+%% \end{itemize}
+
+The \path{dist/install/boot} directory will also contain the config
+files used for building the XenLinux kernels, and also versions of Xen
+and XenLinux kernels that contain debug symbols (\path{xen-syms-2.0.6}
+and \path{vmlinux-syms-2.6.11.11-xen0}) which are essential for
+interpreting crash dumps. Retain these files as the developers may
+wish to see them if you post on the mailing list.
+
+
+\section{Configuration}
+\label{s:configure}
+
+Once you have built and installed the Xen distribution, it is simple
+to prepare the machine for booting and running Xen.
+
+\subsection{GRUB Configuration}
+
+An entry should be added to \path{grub.conf} (often found under
+\path{/boot/} or \path{/boot/grub/}) to allow Xen / XenLinux to boot.
+This file is sometimes called \path{menu.lst}, depending on your
+distribution. The entry should look something like the following:
+
+{\small
+\begin{verbatim}
+title Xen 2.0 / XenLinux 2.6
+ kernel /boot/xen-2.0.gz dom0_mem=131072
+ module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro console=tty0
+\end{verbatim}
+}
+
+The kernel line tells GRUB where to find Xen itself and what boot
+parameters should be passed to it (in this case, setting domain 0's
+memory allocation in kilobytes and the settings for the serial port).
+For more details on the various Xen boot parameters see
+Section~\ref{s:xboot}.
+
+The module line of the configuration describes the location of the
+XenLinux kernel that Xen should start and the parameters that should
+be passed to it (these are standard Linux parameters, identifying the
+root device and specifying it be initially mounted read only and
+instructing that console output be sent to the screen). Some
+distributions such as SuSE do not require the \path{ro} parameter.
+
+%% \framebox{\parbox{5in}{
+%% {\bf Distro specific:} \\
+%% {\it SuSE} --- Omit the {\tt ro} option from the XenLinux
+%% kernel command line, since the partition won't be remounted rw
+%% during boot. }}
+
+
+If you want to use an initrd, just add another \path{module} line to
+the configuration, as usual:
+
+{\small
+\begin{verbatim}
+ module /boot/my_initrd.gz
+\end{verbatim}
+}
+
+As always when installing a new kernel, it is recommended that you do
+not delete existing menu options from \path{menu.lst} --- you may want
+to boot your old Linux kernel in future, particularly if you have
+problems.
+
+\subsection{Serial Console (optional)}
+
+%% kernel /boot/xen-2.0.gz dom0_mem=131072 com1=115200,8n1
+%% module /boot/vmlinuz-2.6-xen0 root=/dev/sda4 ro
+
+
+In order to configure Xen serial console output, it is necessary to
+add an boot option to your GRUB config; e.g.\ replace the above kernel
+line with:
+\begin{quote}
+{\small
+\begin{verbatim}
+ kernel /boot/xen.gz dom0_mem=131072 com1=115200,8n1
+\end{verbatim}}
+\end{quote}
+
+This configures Xen to output on COM1 at 115,200 baud, 8 data bits, 1
+stop bit and no parity. Modify these parameters for your set up.
+
+One can also configure XenLinux to share the serial console; to
+achieve this append ``\path{console=ttyS0}'' to your module line.
+
+If you wish to be able to log in over the XenLinux serial console it
+is necessary to add a line into \path{/etc/inittab}, just as per
+regular Linux. Simply add the line:
+\begin{quote} {\small {\tt c:2345:respawn:/sbin/mingetty ttyS0}}
+\end{quote}
+
+and you should be able to log in. Note that to successfully log in as
+root over the serial line will require adding \path{ttyS0} to
+\path{/etc/securetty} in most modern distributions.
+
+\subsection{TLS Libraries}
+
+Users of the XenLinux 2.6 kernel should disable Thread Local Storage
+(e.g.\ by doing a \path{mv /lib/tls /lib/tls.disabled}) before
+attempting to run with a XenLinux kernel\footnote{If you boot without
+ first disabling TLS, you will get a warning message during the boot
+ process. In this case, simply perform the rename after the machine
+ is up and then run \texttt{/sbin/ldconfig} to make it take effect.}.
+You can always reenable it by restoring the directory to its original
+location (i.e.\ \path{mv /lib/tls.disabled /lib/tls}).
+
+The reason for this is that the current TLS implementation uses
+segmentation in a way that is not permissible under Xen. If TLS is
+not disabled, an emulation mode is used within Xen which reduces
+performance substantially.
+
+We hope that this issue can be resolved by working with Linux
+distribution vendors to implement a minor backward-compatible change
+to the TLS library.
+
+
+\section{Booting Xen}
+
+It should now be possible to restart the system and use Xen. Reboot
+as usual but choose the new Xen option when the Grub screen appears.
+
+What follows should look much like a conventional Linux boot. The
+first portion of the output comes from Xen itself, supplying low level
+information about itself and the machine it is running on. The
+following portion of the output comes from XenLinux.
+
+You may see some errors during the XenLinux boot. These are not
+necessarily anything to worry about --- they may result from kernel
+configuration differences between your XenLinux kernel and the one you
+usually use.
+
+When the boot completes, you should be able to log into your system as
+usual. If you are unable to log in to your system running Xen, you
+should still be able to reboot with your normal Linux kernel.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/introduction.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/introduction.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,143 @@
+\chapter{Introduction}
+
+
+Xen is a \emph{paravirtualising} virtual machine monitor (VMM), or
+`hypervisor', for the x86 processor architecture. Xen can securely
+execute multiple virtual machines on a single physical system with
+close-to-native performance. The virtual machine technology
+facilitates enterprise-grade functionality, including:
+
+\begin{itemize}
+\item Virtual machines with performance close to native hardware.
+\item Live migration of running virtual machines between physical
+ hosts.
+\item Excellent hardware support (supports most Linux device drivers).
+\item Sandboxed, re-startable device drivers.
+\end{itemize}
+
+Paravirtualisation permits very high performance virtualisation, even
+on architectures like x86 that are traditionally very hard to
+virtualise.
+
+The drawback of this approach is that it requires operating systems to
+be \emph{ported} to run on Xen. Porting an OS to run on Xen is
+similar to supporting a new hardware platform, however the process is
+simplified because the paravirtual machine architecture is very
+similar to the underlying native hardware. Even though operating
+system kernels must explicitly support Xen, a key feature is that user
+space applications and libraries \emph{do not} require modification.
+
+Xen support is available for increasingly many operating systems:
+right now, Linux 2.4, Linux 2.6 and NetBSD are available for Xen 2.0.
+A FreeBSD port is undergoing testing and will be incorporated into the
+release soon. Other OS ports, including Plan 9, are in progress. We
+hope that that arch-xen patches will be incorporated into the
+mainstream releases of these operating systems in due course (as has
+already happened for NetBSD).
+
+Possible usage scenarios for Xen include:
+
+\begin{description}
+\item [Kernel development.] Test and debug kernel modifications in a
+ sandboxed virtual machine --- no need for a separate test machine.
+\item [Multiple OS configurations.] Run multiple operating systems
+ simultaneously, for instance for compatibility or QA purposes.
+\item [Server consolidation.] Move multiple servers onto a single
+ physical host with performance and fault isolation provided at
+ virtual machine boundaries.
+\item [Cluster computing.] Management at VM granularity provides more
+ flexibility than separately managing each physical host, but better
+ control and isolation than single-system image solutions,
+ particularly by using live migration for load balancing.
+\item [Hardware support for custom OSes.] Allow development of new
+ OSes while benefiting from the wide-ranging hardware support of
+ existing OSes such as Linux.
+\end{description}
+
+
+\section{Structure of a Xen-Based System}
+
+A Xen system has multiple layers, the lowest and most privileged of
+which is Xen itself.
+
+Xen in turn may host multiple \emph{guest} operating systems, each of
+which is executed within a secure virtual machine (in Xen terminology,
+a \emph{domain}). Domains are scheduled by Xen to make effective use
+of the available physical CPUs. Each guest OS manages its own
+applications, which includes responsibility for scheduling each
+application within the time allotted to the VM by Xen.
+
+The first domain, \emph{domain 0}, is created automatically when the
+system boots and has special management privileges. Domain 0 builds
+other domains and manages their virtual devices. It also performs
+administrative tasks such as suspending, resuming and migrating other
+virtual machines.
+
+Within domain 0, a process called \emph{xend} runs to manage the
+system. \Xend is responsible for managing virtual machines and
+providing access to their consoles. Commands are issued to \xend over
+an HTTP interface, either from a command-line tool or from a web
+browser.
+
+
+\section{Hardware Support}
+
+Xen currently runs only on the x86 architecture, requiring a `P6' or
+newer processor (e.g. Pentium Pro, Celeron, Pentium II, Pentium III,
+Pentium IV, Xeon, AMD Athlon, AMD Duron). Multiprocessor machines are
+supported, and we also have basic support for HyperThreading (SMT),
+although this remains a topic for ongoing research. A port
+specifically for x86/64 is in progress, although Xen already runs on
+such systems in 32-bit legacy mode. In addition a port to the IA64
+architecture is approaching completion. We hope to add other
+architectures such as PPC and ARM in due course.
+
+Xen can currently use up to 4GB of memory. It is possible for x86
+machines to address up to 64GB of physical memory but there are no
+current plans to support these systems: The x86/64 port is the planned
+route to supporting larger memory sizes.
+
+Xen offloads most of the hardware support issues to the guest OS
+running in Domain~0. Xen itself contains only the code required to
+detect and start secondary processors, set up interrupt routing, and
+perform PCI bus enumeration. Device drivers run within a privileged
+guest OS rather than within Xen itself. This approach provides
+compatibility with the majority of device hardware supported by Linux.
+The default XenLinux build contains support for relatively modern
+server-class network and disk hardware, but you can add support for
+other hardware by configuring your XenLinux kernel in the normal way.
+
+
+\section{History}
+
+Xen was originally developed by the Systems Research Group at the
+University of Cambridge Computer Laboratory as part of the XenoServers
+project, funded by the UK-EPSRC.
+
+XenoServers aim to provide a `public infrastructure for global
+distributed computing', and Xen plays a key part in that, allowing us
+to efficiently partition a single machine to enable multiple
+independent clients to run their operating systems and applications in
+an environment providing protection, resource isolation and
+accounting. The project web page contains further information along
+with pointers to papers and technical reports:
+\path{http://www.cl.cam.ac.uk/xeno}
+
+Xen has since grown into a fully-fledged project in its own right,
+enabling us to investigate interesting research issues regarding the
+best techniques for virtualising resources such as the CPU, memory,
+disk and network. The project has been bolstered by support from
+Intel Research Cambridge, and HP Labs, who are now working closely
+with us.
+
+Xen was first described in a paper presented at SOSP in
+2003\footnote{\tt
+ http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf}, and the
+first public release (1.0) was made that October. Since then, Xen has
+significantly matured and is now used in production scenarios on many
+sites.
+
+Xen 2.0 features greatly enhanced hardware support, configuration
+flexibility, usability and a larger complement of supported operating
+systems. This latest release takes Xen a step closer to becoming the
+definitive open source solution for virtualisation.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/redhat.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/redhat.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,61 @@
+\chapter{Installing Xen / XenLinux on Red~Hat or Fedora Core}
+
+When using Xen / XenLinux on a standard Linux distribution there are a
+couple of things to watch out for:
+
+Note that, because domains greater than 0 don't have any privileged
+access at all, certain commands in the default boot sequence will fail
+e.g.\ attempts to update the hwclock, change the console font, update
+the keytable map, start apmd (power management), or gpm (mouse
+cursor). Either ignore the errors (they should be harmless), or
+remove them from the startup scripts. Deleting the following links
+are a good start: {\path{S24pcmcia}}, {\path{S09isdn}},
+{\path{S17keytable}}, {\path{S26apmd}}, {\path{S85gpm}}.
+
+If you want to use a single root file system that works cleanly for
+both domain~0 and unprivileged domains, a useful trick is to use
+different `init' run levels. For example, use run level 3 for
+domain~0, and run level 4 for other domains. This enables different
+startup scripts to be run in depending on the run level number passed
+on the kernel command line.
+
+If using NFS root files systems mounted either from an external server
+or from domain0 there are a couple of other gotchas. The default
+{\path{/etc/sysconfig/iptables}} rules block NFS, so part way through
+the boot sequence things will suddenly go dead.
+
+If you're planning on having a separate NFS {\path{/usr}} partition,
+the RH9 boot scripts don't make life easy - they attempt to mount NFS
+file systems way to late in the boot process. The easiest way I found
+to do this was to have a {\path{/linuxrc}} script run ahead of
+{\path{/sbin/init}} that mounts {\path{/usr}}:
+
+\begin{quote}
+ \begin{small}\begin{verbatim}
+ #!/bin/bash
+ /sbin/ipconfig lo 127.0.0.1
+ /sbin/portmap
+ /bin/mount /usr
+ exec /sbin/init "$@" <>/dev/console 2>&1
+\end{verbatim}\end{small}
+\end{quote}
+
+%% $ XXX SMH: font lock fix :-)
+
+The one slight complication with the above is that
+{\path{/sbin/portmap}} is dynamically linked against
+{\path{/usr/lib/libwrap.so.0}} Since this is in {\path{/usr}}, it
+won't work. This can be solved by copying the file (and link) below
+the {\path{/usr}} mount point, and just let the file be `covered' when
+the mount happens.
+
+In some installations, where a shared read-only {\path{/usr}} is being
+used, it may be desirable to move other large directories over into
+the read-only {\path{/usr}}. For example, you might replace
+{\path{/bin}}, {\path{/lib}} and {\path{/sbin}} with links into
+{\path{/usr/root/bin}}, {\path{/usr/root/lib}} and
+{\path{/usr/root/sbin}} respectively. This creates other problems for
+running the {\path{/linuxrc}} script, requiring bash, portmap, mount,
+ifconfig, and a handful of other shared libraries to be copied below
+the mount point --- a simple statically-linked C program would solve
+this problem.
diff -r c0796e18b6a4 -r 750ad97f37b0 docs/src/user/start_addl_dom.tex
--- /dev/null Tue Sep 20 09:08:26 2005
+++ b/docs/src/user/start_addl_dom.tex Tue Sep 20 09:17:33 2005
@@ -0,0 +1,172 @@
+\chapter{Starting Additional Domains}
+
+The first step in creating a new domain is to prepare a root
+filesystem for it to boot from. Typically, this might be stored in a
+normal partition, an LVM or other volume manager partition, a disk
+file or on an NFS server. A simple way to do this is simply to boot
+from your standard OS install CD and install the distribution into
+another partition on your hard drive.
+
+To start the \xend\ control daemon, type
+\begin{quote}
+ \verb!# xend start!
+\end{quote}
+
+If you wish the daemon to start automatically, see the instructions in
+Section~\ref{s:xend}. Once the daemon is running, you can use the
+\path{xm} tool to monitor and maintain the domains running on your
+system. This chapter provides only a brief tutorial. We provide full
+details of the \path{xm} tool in the next chapter.
+
+% \section{From the web interface}
+%
+% Boot the Xen machine and start Xensv (see Chapter~\ref{cha:xensv}
+% for more details) using the command: \\
+% \verb_# xensv start_ \\
+% This will also start Xend (see Chapter~\ref{cha:xend} for more
+% information).
+%
+% The domain management interface will then be available at {\tt
+% http://your\_machine:8080/}. This provides a user friendly wizard
+% for starting domains and functions for managing running domains.
+%
+% \section{From the command line}
+
+
+\section{Creating a Domain Configuration File}
+
+Before you can start an additional domain, you must create a
+configuration file. We provide two example files which you can use as
+a starting point:
+\begin{itemize}
+\item \path{/etc/xen/xmexample1} is a simple template configuration
+ file for describing a single VM.
+
+\item \path{/etc/xen/xmexample2} file is a template description that
+ is intended to be reused for multiple virtual machines. Setting the
+ value of the \path{vmid} variable on the \path{xm} command line
+ fills in parts of this template.
+\end{itemize}
+
+Copy one of these files and edit it as appropriate. Typical values
+you may wish to edit include:
+
+\begin{quote}
+\begin{description}
+\item[kernel] Set this to the path of the kernel you compiled for use
+ with Xen (e.g.\ \path{kernel = `/boot/vmlinuz-2.6-xenU'})
+\item[memory] Set this to the size of the domain's memory in megabytes
+ (e.g.\ \path{memory = 64})
+\item[disk] Set the first entry in this list to calculate the offset
+ of the domain's root partition, based on the domain ID. Set the
+ second to the location of \path{/usr} if you are sharing it between
+ domains (e.g.\ \path{disk = [`phy:your\_hard\_drive\%d,sda1,w' \%
+ (base\_partition\_number + vmid),
+ `phy:your\_usr\_partition,sda6,r' ]}
+\item[dhcp] Uncomment the dhcp variable, so that the domain will
+ receive its IP address from a DHCP server (e.g.\ \path{dhcp=`dhcp'})
+\end{description}
+\end{quote}
+
+You may also want to edit the {\bf vif} variable in order to choose
+the MAC address of the virtual ethernet interface yourself. For
+example:
+\begin{quote}
+\verb_vif = [`mac=00:06:AA:F6:BB:B3']_
+\end{quote}
+If you do not set this variable, \xend\ will automatically generate a
+random MAC address from an unused range.
+
+
+\section{Booting the Domain}
+
+The \path{xm} tool provides a variety of commands for managing
+domains. Use the \path{create} command to start new domains. Assuming
+you've created a configuration file \path{myvmconf} based around
+\path{/etc/xen/xmexample2}, to start a domain with virtual machine
+ID~1 you should type:
+
+\begin{quote}
+\begin{verbatim}
+# xm create -c myvmconf vmid=1
+\end{verbatim}
+\end{quote}
+
+The \path{-c} switch causes \path{xm} to turn into the domain's
+console after creation. The \path{vmid=1} sets the \path{vmid}
+variable used in the \path{myvmconf} file.
+
+You should see the console boot messages from the new domain appearing
+in the terminal in which you typed the command, culminating in a login
+prompt.
+
+
+\section{Example: ttylinux}
+
+Ttylinux is a very small Linux distribution, designed to require very
+few resources. We will use it as a concrete example of how to start a
+Xen domain. Most users will probably want to install a full-featured
+distribution once they have mastered the basics\footnote{ttylinux is
+ maintained by Pascal Schmidt. You can download source packages from
+ the distribution's home page: {\tt
+ http://www.minimalinux.org/ttylinux/}}.
+
+\begin{enumerate}
+\item Download and extract the ttylinux disk image from the Files
+ section of the project's SourceForge site (see
+ \path{http://sf.net/projects/xen/}).
+\item Create a configuration file like the following:
+\begin{verbatim}
+kernel = "/boot/vmlinuz-2.6-xenU"
+memory = 64
+name = "ttylinux"
+nics = 1
+ip = "1.2.3.4"
+disk = ['file:/path/to/ttylinux/rootfs,sda1,w']
+root = "/dev/sda1 ro"
+\end{verbatim}
+\item Now start the domain and connect to its console:
+\begin{verbatim}
+xm create configfile -c
+\end{verbatim}
+\item Login as root, password root.
+\end{enumerate}
+
+
+\section{Starting / Stopping Domains Automatically}
+
+It is possible to have certain domains start automatically at boot
+time and to have dom0 wait for all running domains to shutdown before
+it shuts down the system.
+
+To specify a domain is to start at boot-time, place its configuration
+file (or a link to it) under \path{/etc/xen/auto/}.
+
+A Sys-V style init script for Red Hat and LSB-compliant systems is
+provided and will be automatically copied to \path{/etc/init.d/}
+during install. You can then enable it in the appropriate way for
+your distribution.
+
+For instance, on Red Hat:
+
+\begin{quote}
+ \verb_# chkconfig --add xendomains_
+\end{quote}
+
+By default, this will start the boot-time domains in runlevels 3, 4
+and 5.
+
+You can also use the \path{service} command to run this script
+manually, e.g:
+
+\begin{quote}
+ \verb_# service xendomains start_
+
+ Starts all the domains with config files under /etc/xen/auto/.
+\end{quote}
+
+\begin{quote}
+ \verb_# service xendomains stop_
+
+ Shuts down ALL running Xen domains.
+\end{quote}
_______________________________________________
Xen-changelog mailing list
Xen-changelog@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-changelog
|
Home