Re: [Xen-devel] Building Xen Test Days into the conclusion of the Xen 4.3 release cycle

[George Dunlap <dunlapg@xxxxxxxxx>]

On Thu, Apr 25, 2013 at 12:18 PM, Lars Kurth <lars.kurth@xxxxxxx> wrote:
> On 25/04/2013 09:25, Lars Kurth wrote:
>
>>> Views? Should I start announcing the dates and promote the Test Days?
>> All sounds good, I think you should go ahead.
> Any objections?
>
> Appears there are none. I created
> - http://wiki.xen.org/wiki/Xen_4.3_RC1_test_instructions which needs to be
> popolated
> - Remember Mondy is Docs Day - A perfect opportunity to do this
> - Also updated http://wiki.xen.org/wiki/Xen_Test_Days

I was thinking it would be good to think about a "test matrix".  Below
is a sort of summary of what I came up with -- I'd appreciate if
people could add things I'm missing.

In general I was thinking "variables" as a sort of "state of the
system" V, and then we want to test specific functionality f given
state V.

 -George

Variables
 - Host Hardware
   - CPU (Intel, AMD)
   - NUMA topology
   - Amount of memory
   - IOMMU, &c
 - Host config
   - Domain 0 distro
     { Debian, Fedora, Ubuntu, Arch, Alpine, NetBSD }
   - Domain 0 kernel
     { distro, mainline most recent, mainline stable }
   - Network driver domain
   - Networking (bridging / OVS / &c)
   - With / without default XSM policy
 - Guest config
   - Guest OS
   - PV / PVHVM (for Linux / NetBSD guests)
   - qemu version ( qemu-xen-traditional / qemu-xen / straight qemu )
   - Stubdoms
   - Emualted / PV devices (disk, network)
   - USB pass-through

Functionality
 - Build / install
 - Basic
   create, list, shutdown, reboot, destroy, suspend/resume, migrate,
pause, unpause, console
 - Extended basic
   cd-insert, cd-eject, button-press, vcpu-pin, vcpu-set, domid,
domname, rename, trigger, sysrq,
   info, dmesg, top, config-update, info
 - Guest devices
   - PCI pass-through
   - Network / block attach
   - Host usb (config file only for 4.3)
   - qxl spice
 - Memory
   - Ballooning
     PoD, mem-max, mem-set
   - Page sharing
   - Hypervisor swap
 - Scheduling parameters
 - NUMA
 - debug-keys
 - vtpm
 - tmem
 - cpupools
 - remus
 - alternate block / network scripts
 - Flask

Types of testing
 - Smoke-test
   Just do something quickly to see if it works
 - Normal functional test
   Try to use it the way you expect it to work
 - Stress-testing
   Automated heavy use over long period of time
 - Edge-case testing
   Try to use it in ways you don't expect it to be used
 - Performance
   Test it specifically measuring the performance

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel