[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Nested virtualization instabilities



Etzion Bar-Noy wrote on 2013-12-20:
> Sorry for spamming. Actually - it is a bit different - two results
> which appear (on XenServer and on VMware). The previous messages were
> for Vmware. The one below is for XenServer:

I guess you are using shadow mode for L1 VMM. With shadow L1, this is a known 
issue to me. Can you try to use L1 EPT to see whether it is reproducible?

> 
> xl dmesg:
> 
> (XEN) stdvga.c:151:d10 leaving stdvga (XEN) common.c:3525:d0 tracking
> VRAM f0000 - f0800 (XEN) stdvga.c:147:d10 entering stdvga and caching
> modes (XEN) realmode.c:115:d10 Failed to emulate insn. (XEN)
> realmode.c:167:d10 Real-mode emulation failed @ f000:0000fff0: 00 90 23
> 00 c0 82 (XEN) domain_crash called from realmode.c:168 (XEN) Domain 10
> (vcpu#0) crashed on cpu#8: (XEN) ----[ Xen-4.4-unstable  x86_64  debug=y
>  Not tainted ]---- (XEN) CPU:    8 (XEN) RIP:   
> f000:[<000000000000fff0>] (XEN) RFLAGS: 0000000000000002   CONTEXT: hvm
> guest (XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx:
> 0000000000000000 (XEN) rdx: 00000000078bfbff   rsi: 0000000000000000  
> rdi: 0000000000000000 (XEN) rbp: 0000000000000000   rsp:
> 0000000000000000   r8: 0000000000000000 (XEN) r9:  0000000000000000  
> r10: 0000000000000000   r11: 0000000000000000 (XEN) r12:
> 0000000000000000   r13: 0000000000000000   r14: 0000000000000000 (XEN)
> r15: 0000000000000000   cr0: 0000000000000030   cr4: 0000000000002040
> (XEN) cr3: 0000000000000000   cr2: 0000000000000000 (XEN) ds: 0000   es:
> 0000   fs: 0000   gs: 0000   ss: 0000   cs: f000 (XEN) <vm_launch_fail>
> error code 7 (XEN) domain_crash_sync called from vmcs.c:1293 (XEN)
> Domain 9 (vcpu#0) crashed on cpu#5: (XEN) ----[ Xen-4.4-unstable  x86_64
>  debug=y  Not tainted ]---- (XEN) CPU:    5 (XEN) RIP:   
> 0000:[<0000000000000000>] (XEN) RFLAGS: 0000000000000002   CONTEXT: hvm
> guest (XEN) rax: 0000000000000000   rbx: ffff830824f90000   rcx:
> ffff830824f97f80 (XEN) rdx: ffff82d0801d0cb9   rsi: 0000000000000000  
> rdi: ffff82d0801de7ec (XEN) rbp: ffff82d080105ab1   rsp:
> 0000000000000000   r8: 0000000000000004 (XEN) r9:  ffff82d080105b20  
> r10: ffff830824f97f70   r11: 0000000000000000 (XEN) r12:
> ffff830824f97f50   r13: ffff83080191c000   r14: 0000000000000004 (XEN)
> r15: ffff82d080187e15   cr0: 0000000000000039   cr4: 0000000000002050
> (XEN) cr3: 00000000feffa000   cr2: 0000000000000000 (XEN) ds: 0000   es:
> 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0000 (XEN)
> grant_table.c:289:d0 Increased maptrack size to 2 frames (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22 (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
> 
> 
> Thanks
> Etzion
> 
> 
> 
> On Thu, Dec 19, 2013 at 9:40 PM, Etzion Bar-Noy <etzion@xxxxxxxxxxxx> wrote:
> 
> 
>       Hi.
> 
>       With the added flag suggested by you, the VM (nested hypervisor) 
> crashed.
> Now ESXi will attempt to start the VM and crash as well (unlike
> before, where it just failed to start it with 'unknown error').
> 
>       XenServer 6.2 still crashes, just the same.
> 
> 
>       I am attempting to deploy a Centos6 machine (and clone it) so that I
> will be able to test both KVM and Xen community (latest stable RPM
> release for Centos6).
> 
> 
>       xl dmesg looks much like before:
> 
>       (XEN) stdvga.c:151:d12 leaving stdvga
>       (XEN) common.c:3525:d0 tracking VRAM f0000 - f0800
>       (XEN) stdvga.c:147:d12 entering stdvga and caching modes
>       (XEN) realmode.c:115:d12 Failed to emulate insn.
>       (XEN) realmode.c:167:d12 Real-mode emulation failed @ f000:0000fff0:
> 00 10 23 00 c0 82
>       (XEN) domain_crash called from realmode.c:168
>       (XEN) Domain 12 (vcpu#0) crashed on cpu#4:
> 
>       (XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not tainted ]----
>       (XEN) CPU:    4
> 
>       (XEN) RIP:    f000:[<000000000000fff0>]
> 
>       (XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest
> 
>       (XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx:
> 0000000000000000      (XEN) rdx: 00000000078bfbff   rsi: 0000000000000000  
> rdi: 0000000000000000         (XEN) rbp: 0000000000000000   rsp:
> 0000000000000000   r8: 0000000000000000       (XEN) r9:  0000000000000000  
> r10: 0000000000000000   r11: 0000000000000000         (XEN) r12:
> 0000000000000000   r13: 0000000000000000   r14: 0000000000000000      (XEN)
> r15: 0000000000000000   cr0: 0000000000000030   cr4: 0000000000002040
>       (XEN) cr3: 0000000000000000   cr2: 0000000000000000     (XEN) ds: 0000  
> es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: f000  (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22         (XEN)
> event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
> 
> 
>       Thanks!
>       Etzion
> 
> 
> 
>       On Wed, Dec 18, 2013 at 11:46 AM, Etzion Bar-Noy <etzion@xxxxxxxxxxxx>
> wrote:
> 
> 
>               Hi Mr. Yang.
> 
>               I did attempt to use your suggested configuration with the 
> XenServer
> VM, but to no avail. The L2 VM keeps on crushing. I defined it now to
> leave a dump file when crushing, and if you desire so, I will find
> some method of uploading it for your viewing pleasure. As I am not a
> programmer, but an integrator, my abilities in this direction are limited at 
> best.
> 
>               The L3 VM type is 'generic VM', because I wanted to enforce the
> system to use HVM emulation.
> 
> 
>               I have just applied the patch you supplied me to my 'staging' 
> git
> checkout, and just finished compiling it. I will soon apply the newly
> created RPMs on my test (and hopefully - to be production soon)
> server, and will check VMware.
> 
> 
>               Thanks for the input. I will try and then respond with results.
> 
>               Etzion
> 
> 
> 
>               On Wed, Dec 18, 2013 at 3:33 AM, Zhang, Yang Z
> <yang.z.zhang@xxxxxxxxx> wrote:
> 
> 
>                       Etzion Bar-Noy wrote on 2013-12-16:
>                       > Hi.
>                       >
> 
>                       Hi Etzion Bar-Noy
> 
>                       It's glad to know someone is using Nested Xen and 
> report bugs to
> us. Also, if there are more nested bugs you found, please CC me too.
> 
> 
>                       >                       > I am using the unstable 
> build-from-source of Xen community. It
> reports                       > as 4.4, and was pulled from Git few days ago. 
> The problem I
> will                  > describe below appears on version 4.3.1, compiled 
> from source
> as well.                      > In both cases - I used spec file to create my 
> own RPMs,
> because it's the right method of doing it.                    >               
>         >                       > Problem: When
> VM running under HVM which is a hypervisor (XenServer                         
> > 6.2 or VMware
> ESXi5.5) attempts to start a (HVM) virtual machine, the                       
> > following
> happens, 100% reproducible on this server.                    >               
>         > XenServer: domain
> crash. XenServer restarts itself. It doesn't happen                   > for 
> PV on
> XenServer VM                  >                       > VMware: An "unknown 
> failure", which leads me to
> think that the nested                         > virtualization properties are 
> not forwarded
> correctly to DomU
> 
> 
>                       To using nested, you need to add the set bit 31 of 
> cupid.1.ecx to
> 0, add the following to your config file:
>                       cpuid = ['0x1:ecx=0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx']
> 
>                       For VMware ESX server, the attached patch is missing in 
> current Xen
> which will caused the guest boot up fail. Also, I know some windows
> guests will boot fail on ESX.
>                       I never tried XenServer. But latest Xen on Xen is 
> working well. So
> can you try the upsteam Xen to see whether it fails too?
> 
>                       BTW, which L2 VM boot fail?
> 
> 
>                       >                       >                       > Just 
> to clarify:                      >                       > Physical hardware: 
> host                       >
>                       > Nested hypervisor: level 1 virtualization: XenServer 
> or VMware ESXi
>                       >                       > Nested VM: level 2 
> virtualization: some generic VM inside
> XenServer or                  > ESXi (level 1)                        >       
>                 >                       > This is the
> configuration used for XS:                    > name = "XenServer"            
>         > uuid =
> "a0b8c668-afaf-887a-1f4b-856dac4c8401"                        > maxmem = 4096 
>                         > memory =
> 4096                  > vcpus = 2                     > builder = "hvm"       
>                 > boot = "c"                    > pae = 1
>                       > acpi = 1                      > apic = 1              
>         > hap = 0                       > viridian = 0                  >
> rtc_timeoffset = 0                    > localtime = 0                         
> > on_poweroff = "destroy"                       >
> on_reboot = "restart"                         > on_crash = "restart"          
>         > sdl = 0                       > vnc =
> 1                     > vncunused = 1                         > keymap = 
> "en-us"                      > disk = [
> "tap2:aio:/dev/share/VMS/xenserver01.lun,hda,w",                      > 
> ",hdc:cdrom,r" ]
> vif = [                       >
> "mac=00:16:3e:14:0d:72,bridge=lan,script=vif-bridge,model=e1000" ]
> parallel = "none"                     > serial = "pty"                        
> > nestedhvm = 1                         >                       > nx=1
>                       >                       >                       > # A 
> note - this problem appears with and without the 'nx'
> flag.                         >                       > A snippet from strace 
> to qemu process:                        >                       >
> poll([{fd=30, events=POLLIN|POLLERR|POLLHUP}, {fd=18,                         
> >
> events=POLLIN|POLLERR|POLLHUP}, {fd=29,                       >
> events=POLLIN|POLLERR|POLLHUP}, {fd=21,                       >
> events=POLLIN|POLLERR|POLLHUP}, {fd=15,                       >
> events=POLLIN|POLLERR|POLLHUP}, {fd=5, events=
> 
>                       > POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN}, 
> {fd=11,
> POLLIN|POLLERR|events=POLLIN}, {fd=28, events=POLLIN}, {fd=6,
> POLLIN|POLLERR|events=POLLIN}], 10, 307) = ?
>                       > ERESTART_RESTARTBLOCK (To POLLIN|POLLERR|be restarted)
> --- SIGHUP
> 
>                       > (Hangup) @ 0 (0) --- write(6, "\1\0\0\0\0\0\0\0", 8) 
> = 8                      >
> rt_sigreturn(0x2)                       = -1 EINTR (Interrupted system
>                       > call) gettimeofday({1386761621, 705266}, NULL) = 0    
>                 >
> clock_gettime(CLOCK_MONOTONIC, {1840, 693236049}) = 0                         
> >
> clock_gettime(CLOCK_MONOTONIC, {1840, 693333096}) = 0                         
> >
> gettimeofday({1386761621, 705546}, NULL) = 0 write(2, "qemu: terminating
>                       > on signal 1", 29) = 29 write(2, " from pid 6933\n", 
> 15) = 15                  >
>                       >                       > From the xl log for this 
> domain:                      > Waiting for domain
> XenServer (domid 5) to die [pid 6933] Domain 5 has                    > shut 
> down,
> reason code 3 0x3 Action for shutdown reason code 3 is                        
> > restart
> Domain                        > 5 needs to be cleaned up: destroying the 
> domain Done.
> Rebooting now                         > Waiting for domain XenServer (domid 
> 6) to die [pid
> 6933]                         >                       >                       
> > From 'xl dmesg' (XEN) <vm_launch_fail> error code 7
> (XEN)                         > domain_crash_sync called from vmcs.c:1293 
> (XEN) Domain 8
> (vcpu#0)                      > crashed on cpu#4: (XEN) ----[ 
> Xen-4.4-unstable  x86_64
> debug=y Not                   > tainted ]---- (XEN) CPU:    4 (XEN) RIP:
> 0000:[<0000000000000000>]                     > (XEN) RFLAGS: 
> 0000000000000002   CONTEXT:
> hvm guest (XEN) rax:                  > 0000000000000000   rbx: 
> ffff830824f98000  
> rcx: ffff830824f9ff80 (XEN)                   > rdx: ffff82d0801d0cf9   rsi:
> 0000000000000000   rdi: ffff82d0801de7fc                      > (XEN) rbp:
> ffff82d080105ab1   rsp: 0000000000000000   r8:                        > 
> 0000000000000004
> (XEN) r9:  ffff82d080105b20   r10: ffff830824f9ff70                   > r11:
> 0000000000000000 (XEN) r12: ffff830824f9ff50   r13:                   >
> ffff830654f9e000   r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43
>                       > cr0: 0000000000000039   cr4: 0000000000002050 (XEN) 
> cr3:                      >
> 00000000feffa000   cr2: 0000000000000000 (XEN) ds: 0000 es: 0000   fs:
>                       > 0000   gs: 0000   ss: 0000   cs: 0000 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader 
> (d9) Detected
>                       > Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, 
> event channel 4
> (d9)                  > System requested SeaBIOS (d9) CPU speed is 3400 MHz 
> (d9)
> Relocating                    > guest memory for lowmem MMIO space disabled 
> (XEN)
> irq.c:270: Dom9 PCI                   > link 0 changed 0 -> 5 (d9) PCI-ISA 
> link 0
> routed to IRQ5 (XEN)                  > irq.c:270: Dom9 PCI link 1 changed 0 
> -> 10
> (d9) PCI-ISA link 1 routed                    > to IRQ10 (XEN)                
>         > irq.c:270: Dom9 PCI
> link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed                     > to    
>                 > IRQ11
> (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA                  
> > link
>                       > 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) 
> pci dev 02:0
>                       > INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in 
> high memory;
>                       > setting high_mem resource base to 10f800000 (d9) pci 
> dev 03:0 bar
> 10                    > size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 
> size
> 001000000:                    > 0f2000008 (d9) pci dev 04:0 bar 30 size 
> 000040000:
> 0f3000000 (d9) pci                    > dev 04:0 bar 10 size 000020000: 
> 0f3040000 (d9)
> pci dev 03:0 bar 30                   > size                  > 000010000: 
> 0f3060000 (d9) pci dev
> 03:0 bar 14 size 000001000:                   > 0f3070000                     
> > (d9) pci dev 02:0 bar 10
> size 000000100: 00000c001 (d9) pci dev 04:0                   > bar           
>         > 14 size
> 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010:                 
>         >
> 00000c141 (d9) Multiprocessor initialisation: (d9)  - CPU0 ... 46-bit
>                       > phys ... fixed MTRRs ... var MTRRs [2/8] ... done. 
> (d9)  - CPU1 ...
>                       > 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... 
> done. (d9)
> Testing                       > HVM environment: (d9)  - REP INSB across page 
> boundaries
> ... passed                    > (d9)                  >  - GS base MSRs and 
> SWAPGS ... passed (d9)
> Passed 2 of 2 tests (d9)                      > Writing SMBIOS tables ... 
> (d9) Loading
> SeaBIOS ... (d9) Creating MP                  > tables ... (d9) Loading ACPI 
> ... (d9)
> vm86 TSS at fc00a080 (d9) BIOS                        > map: (d9)  
> 10000-100d3: Scratch
> space (d9)  e0000-fffff: Main BIOS                    > (d9)                  
> > E820 table: (d9) 
> [00]: 00000000:00000000 - 00000000:000a0000: RAM                      > (d9)  
>                 > HOLE:
> 00000000:000a0000 - 00000000:000e0000 (d9)  [01]:                     > 
> 00000000:000e0000
> - 00000000:00100000: RESERVED (d9) [02]:                      > 
> 00000000:00100000 -
> 00000000:f0000000: RAM (d9)  HOLE:                    > 00000000:f0000000     
>                 > -
> 00000000:fc000000 (d9)  [03]: 00000000:fc000000 - 00000001:00000000:
>                       > RESERVED (d9)  [04]: 00000001:00000000 - 
> 00000001:0f800000: RAM
> (d9)                  > Invoking SeaBIOS ... (d9) SeaBIOS (version            
>         >
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen
>                       > hypervisor signature at 40000000 (d9) xen: copy 
> e820... (d9)                  >
> Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU           
>         >
> Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated
>                       > Xen hypercall page at effff000 (d9) Detected Xen 
> v4.4-unstable (d9)
>                       > xen: copy BIOS tables... (d9) Copying SMBIOS entry 
> point from                         >
> 0x00010010 to 0x000f1020                      > (d9) Copying MPTABLE from
> 0xfc001170/fc001180 to 0x000f0f20 (d9)                        > Copying PIR 
> from 0x00010030
> to 0x000f0ea0 (d9) Copying ACPI RSDP from                     > 0x000100b0 to 
> 0x000f0e70
> (d9) Using pmtimer, ioport 0xb008, freq 3579                  > kHz (d9) Scan 
> for VGA
> option rom (d9) WARNING! Found unaligned PCI rom                      > 
> (vd=1013:00b8)
>                       > (d9) Running option rom at c000:0003 (XEN) 
> stdvga.c:147:d9 entering
>                       > stdvga and caching modes (d9) Turning on vga text 
> mode console (d9)
>                       > SeaBIOS (version 
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm)
>                       > (d9) Machine UUID 
> a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0
>                       > lpt ports                     > (d9) Found 1 serial 
> ports (d9) ATA controller 1 at
> 1f0/3f4/c140 (irq                     > 14 dev 9) (d9) ATA controller 2 at 
> 170/374/c148
> (irq 15 dev 9) (d9) ata0-0:                   > QEMU HARDDISK ATA-7 Hard-Disk 
> (14336
> MiBytes) (d9) Searching bootorder                     > for:
> /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM             
>         >
> ATAPI-4 DVD/CD] (d9) Searching bootorder for:                         >
> /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All
>                       > threads complete. (d9) Scan for option roms (d9) 
> Running option rom
> at                    > c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 
> (d9) pmm
> call                  > arg1=1                        > (d9) pmm call arg1=0 
> (d9) Searching bootorder for:
> /pci@i0cf8/*@4 (d9)                   > (d9) Press F12 for boot menu. (d9) 
> (d9)
> Searching bootorder for: HALT                         > (d9) drive 0x000f0e20:
> PCHS=16383/16/63 translation=lba                      > LCHS=1024/255/63      
>                 > s=29360128
> (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9)                   
> >
> Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9)   0:
>                       > 0000000000000000 - 000000000009fc00 = 1 RAM (d9)   1:
> 000000000009fc00 -                    > 00000000000a0000 = 2 RESERVED (d9)   
> 2:
> 00000000000f0000 -                    > 0000000000100000 = 2 RESERVED (d9)   
> 3:
> 0000000000100000 -                    > 00000000effff000 = 1 RAM (d9)   4:
> 00000000effff000 - 00000000f0000000 =                         > 2 RESERVED 
> (d9)   5:
> 00000000fc000000 - 0000000100000000 = 2 RESERVED                      > (d9)  
>  6:
> 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter                        
> > handle_19:
> (d9)   NULL (d9) Booting from Hard Disk... (d9) Booting from                  
> >
> 0000:7c00 [root@monster ~]# xl dmesg (XEN) <vm_launch_fail> error code
>                       > 7                     > (XEN) domain_crash_sync 
> called from vmcs.c:1293 (XEN) Domain
> 8                     > (vcpu#0) crashed on cpu#4: (XEN) ----[ 
> Xen-4.4-unstable x86_64 
> debug=y  Not                  > tainted ]---- (XEN) CPU:    4 (XEN) RIP:
> 0000:[<0000000000000000>]                     > (XEN) RFLAGS: 
> 0000000000000002   CONTEXT:
> hvm guest (XEN) rax:                  > 0000000000000000   rbx: 
> ffff830824f98000  
> rcx: ffff830824f9ff80 (XEN)                   > rdx: ffff82d0801d0cf9   rsi:
> 0000000000000000   rdi: ffff82d0801de7fc                      > (XEN) rbp:
> ffff82d080105ab1   rsp: 0000000000000000   r8:                        > 
> 0000000000000004
> (XEN) r9:  ffff82d080105b20   r10: ffff830824f9ff70                   > r11:
> 0000000000000000 (XEN) r12: ffff830824f9ff50   r13:                   >
> ffff830654f9e000   r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43
>                       > cr0: 0000000000000039   cr4: 0000000000002050 (XEN) 
> cr3:                      >
> 00000000feffa000   cr2: 0000000000000000 (XEN) ds: 0000 es: 0000   fs:
>                       > 0000   gs: 0000   ss: 0000   cs: 0000 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (XEN) 
> event_channel.c:270:d0
>                       > EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader 
> (d9) Detected
>                       > Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, 
> event channel 4
> (d9)                  > System requested SeaBIOS (d9) CPU speed is 3400 MHz 
> (d9)
> Relocating                    > guest memory for lowmem MMIO space disabled 
> (XEN)
> irq.c:270: Dom9 PCI                   > link 0 changed 0 -> 5 (d9) PCI-ISA 
> link 0
> routed to IRQ5 (XEN)                  > irq.c:270: Dom9 PCI link 1 changed 0 
> -> 10
> (d9) PCI-ISA link 1 routed                    > to IRQ10 (XEN)                
>         > irq.c:270: Dom9 PCI
> link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed                     > to    
>                 > IRQ11
> (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA                  
> > link
>                       > 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) 
> pci dev 02:0
>                       > INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in 
> high memory;
>                       > setting high_mem resource base to 10f800000 (d9) pci 
> dev 03:0 bar
> 10                    > size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 
> size
> 001000000:                    > 0f2000008 (d9) pci dev 04:0 bar 30 size 
> 000040000:
> 0f3000000 (d9) pci                    > dev 04:0 bar 10 size 000020000: 
> 0f3040000 (d9)
> pci dev 03:0 bar 30                   > size                  > 000010000: 
> 0f3060000 (d9) pci dev
> 03:0 bar 14 size 000001000:                   > 0f3070000                     
> > (d9) pci dev 02:0 bar 10
> size 000000100: 00000c001 (d9) pci dev 04:0                   > bar           
>         > 14 size
> 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010:                 
>         >
> 00000c141 (d9) Multiprocessor initialisation: (d9)  - CPU0 ... 46-bit
>                       > phys ... fixed MTRRs ... var MTRRs [2/8] ... done. 
> (d9)  - CPU1 ...
>                       > 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... 
> done. (d9)
> Testing                       > HVM environment: (d9)  - REP INSB across page 
> boundaries
> ... passed                    > (d9)                  >  - GS base MSRs and 
> SWAPGS ... passed (d9)
> Passed 2 of 2 tests (d9)                      > Writing SMBIOS tables ... 
> (d9) Loading
> SeaBIOS ... (d9) Creating MP                  > tables ... (d9) Loading ACPI 
> ... (d9)
> vm86 TSS at fc00a080 (d9) BIOS                        > map: (d9)  
> 10000-100d3: Scratch
> space (d9)  e0000-fffff: Main BIOS                    > (d9)                  
> > E820 table: (d9) 
> [00]: 00000000:00000000 - 00000000:000a0000: RAM                      > (d9)  
>                 > HOLE:
> 00000000:000a0000 - 00000000:000e0000 (d9)  [01]:                     > 
> 00000000:000e0000
> - 00000000:00100000: RESERVED (d9) [02]:                      > 
> 00000000:00100000 -
> 00000000:f0000000: RAM (d9)  HOLE:                    > 00000000:f0000000     
>                 > -
> 00000000:fc000000 (d9)  [03]: 00000000:fc000000 - 00000001:00000000:
>                       > RESERVED (d9)  [04]: 00000001:00000000 - 
> 00000001:0f800000: RAM
> (d9)                  > Invoking SeaBIOS ... (d9) SeaBIOS (version            
>         >
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen
>                       > hypervisor signature at 40000000 (d9) xen: copy 
> e820... (d9)                  >
> Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU           
>         >
> Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated
>                       > Xen hypercall page at effff000 (d9) Detected Xen 
> v4.4-unstable (d9)
>                       > xen: copy BIOS tables... (d9) Copying SMBIOS entry 
> point from                         >
> 0x00010010 to 0x000f1020                      > (d9) Copying MPTABLE from
> 0xfc001170/fc001180 to 0x000f0f20 (d9)                        > Copying PIR 
> from 0x00010030
> to 0x000f0ea0 (d9) Copying ACPI RSDP from                     > 0x000100b0 to 
> 0x000f0e70
> (d9) Using pmtimer, ioport 0xb008, freq 3579                  > kHz (d9) Scan 
> for VGA
> option rom (d9) WARNING! Found unaligned PCI rom                      > 
> (vd=1013:00b8)
>                       > (d9) Running option rom at c000:0003 (XEN) 
> stdvga.c:147:d9 entering
>                       > stdvga and caching modes (d9) Turning on vga text 
> mode console (d9)
>                       > SeaBIOS (version 
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm)
>                       > (d9) Machine UUID 
> a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0
>                       > lpt ports                     > (d9) Found 1 serial 
> ports (d9) ATA controller 1 at
> 1f0/3f4/c140 (irq                     > 14 dev 9) (d9) ATA controller 2 at 
> 170/374/c148
> (irq 15 dev 9) (d9) ata0-0:                   > QEMU HARDDISK ATA-7 Hard-Disk 
> (14336
> MiBytes) (d9) Searching bootorder                     > for:
> /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM             
>         >
> ATAPI-4 DVD/CD] (d9) Searching bootorder for:                         >
> /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All
>                       > threads complete. (d9) Scan for option roms (d9) 
> Running option rom
> at                    > c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 
> (d9) pmm
> call                  > arg1=1                        > (d9) pmm call arg1=0 
> (d9) Searching bootorder for:
> /pci@i0cf8/*@4 (d9)                   > (d9) Press F12 for boot menu. (d9) 
> (d9)
> Searching bootorder for: HALT                         > (d9) drive 0x000f0e20:
> PCHS=16383/16/63 translation=lba                      > LCHS=1024/255/63      
>                 > s=29360128
> (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9)                   
> >
> Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9)   0:
>                       > 0000000000000000 - 000000000009fc00 = 1 RAM (d9)   1:
> 000000000009fc00 -                    > 00000000000a0000 = 2 RESERVED (d9)   
> 2:
> 00000000000f0000 -                    > 0000000000100000 = 2 RESERVED (d9)   
> 3:
> 0000000000100000 -                    > 00000000effff000 = 1 RAM (d9)   4:
> 00000000effff000 - 00000000f0000000 =                         > 2 RESERVED 
> (d9)   5:
> 00000000fc000000 - 0000000100000000 = 2 RESERVED                      > (d9)  
>  6:
> 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter                        
> > handle_19:
> (d9)   NULL (d9) Booting from Hard Disk... (d9) Booting from                  
> >
> 0000:7c00                     >                       >                       
> > System description:                   >                       > 1. Centos 
> 6.4
> x86_64, latest updates                        >                       > 2. 
> Xen4centos packages - current kernel
> is                    > 3.10.23-11.el6.centos.alt.x86_64                      
> >                       > 3. System with 32GB
> RAM                   >                       > 4. Virtual disks are ZFS 
> volumes (aka - block devices).
> Same problem                  > appeared when on local ext3 disks             
>         >                       > 5. Using
> OpenVSwitch version 2.0, compiled to RPM, again.                      >       
>                 > 6. Intel CPU:
>                       > processor    : 0                      > vendor_id    
> : GenuineIntel                   > cpu family  
>  : 6                  > model        : 62                     > model name    
> : Intel(R) Core(TM)
> i7-4930K CPU @ 3.40GHz                        > stepping    : 4               
>         > microcode    : 0x416
>                       > cpu MHz        : 3400.158                     > cache 
> size    : 12288 KB                      >
> physical id    : 0                    > siblings    : 12                      
> > core id        : 0                    >
> cpu cores    : 6                      > apicid        : 0                     
> > initial apicid    : 0                         >
> fpu        : yes                      > fpu_exception    : yes                
>         > cpuid level    : 13
>                       > wp        : yes                       > flags        
> : fpu de tsc msr pae mce cx8 apic
> sep mca cmov pat clflush acpi                         > mmx fxsr sse sse2 ss 
> ht syscall nx lm
> constant_tsc rep_good nopl                    > nonstop_tsc eagerfpu pni 
> pclmulqdq
> monitor est ssse3 cx16 sse4_1                         > sse4_2 popcnt 
> tsc_deadline_timer aes
> xsave avx f16c rdrand hypervisor                      > lahf_lm ida arat epb 
> xsaveopt pln
> pts dtherm fsgsbase erms                      > bogomips    : 6800.31         
>                 > clflush size   
> : 64                  > cache_alignment    : 64                       > 
> address sizes    : 46 bits
> physical, 48 bits virtual                     > power management:             
>         >                       >                       > It was
> suggested by Mr. Ian Campbell to CC both Mr. Jun Nakajima and Mr.             
>         >
> Eddie Dong, so, my apologies if this action is improper.                      
> >                       >                       >
> Any tip or advice would be great. Any request for further info would
>                       > be also great!                        >               
>         > Thanks!                       >                       > Etzion 
> Bar-Noy
> 
> 
> 
>                       Best regards,
>                       Yang
> 
> 
> 
> 
>


Best regards,
Yang



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.