[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] ARM: EXYNOS 5410 - DOM0 not being scheduled



Hello Julien and Ian,
   I at last got some output on the console (with some random first
characters), by doing the below change:
drivers/tty/hvc/hvc_xen.c:
- call dom0_write_console(0, str, len) directly.. ie, remove the if
(xen_domain()) condition. So, this is another issue, why is it not
recognized as a xen domain ?

With the above change, I can see the crash of dom0 as below! Have to
figure out why the architected timer frequency is not available in
dom0!

----------- dom0 output --------------
6Booting Linux on physical CPU 0x0
5Linux version 3.13.0+ (suriyan@Stealth) (gcc version 4.7.3
(Buildroot 2014.02) ) #2 SMP Mon Mar 17 11:02:36 PDT 2014
5Kernel was built at commit id 'aa5c722'
CPU: ARMv7 Processor [412fc0f3] revision 3 (ARMv7), cr=50c5387d
CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
6Machine model: Hardkernel odroid-xu board based on EXYNOS5410
6cma: CMA: reserved 128 MiB at 98000000
6Memory policy: Data cache writealloc
CPU EXYNOS5410 (id 0xe5410023)
7On node 0 totalpages: 131072
7  Normal zone: 1024 pages used for memmap
7  Normal zone: 0 pages reserved
7  Normal zone: 131072 pages, LIFO batch:31
6PERCPU: Embedded 9 pages/cpu @c0c3d000 s15232 r8192 d13440 u36864
7pcpu-alloc: s15232 r8192 d13440 u36864 alloc=9*4096
7pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130048
5Kernel command line: console=hvc0 root=/dev/mmcblk0p2 rw rootwait
earlyprintk=xen
6PID hash table entries: 2048 (order: 1, 8192 bytes)
6Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
6Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 379932K/524288K available (4779K kernel code, 394K rwdata,
2252K rodata, 402K init, 497K bss, 144356K reserved, 0K highmem)
5Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
    vmalloc : 0xe0800000 - 0xff000000   ( 488 MB)
    lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf800000 - 0xbfe00000   (   6 MB)
      .text : 0xc0008000 - 0xc06e5ef4   (7032 kB)
      .init : 0xc06e6000 - 0xc074ab80   ( 403 kB)
      .data : 0xc074c000 - 0xc07ae8a0   ( 395 kB)
       .bss : 0xc07ae8a0 - 0x6Hierarchical RCU implementation.
6    RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
6NR_IRQS:16 nr_irqs:16 16
4Architected timer frequency not available
Division by zero in kernel.
dCPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0+ #2
[<c0012315>] (unwind_backtrace+0x1/0x9c) from [<c000fbf1>]
(show_stack+0x11/0x14)
[<c000fbf1>] (show_stack+0x11/0x14) from [<c0478d5f>] (dump_stack+0x4f/0x64)
[<c0478d5f>] (dump_stack+0x4f/0x64) from [<c025ee7f>] (Ldiv0_64+0x9/0x1a)
[<c025ee7f>] (Ldiv0_64+0x9/0x1a) from [<c0058f65>]
(clockevents_config+0x1d/0x60)
[<c0058f65>] (clockevents_config+0x1d/0x60) from [<c0058fbb>]
(clockevents_config_and_register+0x13/0x1c)
[<c0058fbb>] (clockevents_config_and_register+0x13/0x1c) from
[<c038f1db>] (arch_timer_setup+0x67/0x124)
[<c038f1db>] (arch_timer_setup+0x67/0x124) from [<c070848d>]
(arch_timer_init+0xf5/0x160)
[<c070848d>] (arch_timer_init+0xf5/0x160) from [<c0707d21>]
(clocksource_of_init+0x25/0x40)
[<c0707d21>] (clocksource_of_init+0x25/0x40) from [<c06e67bb>]
(start_kernel+0x187/0x310)
[<c06e67bb>] (start_kernel+0x187/0x310) from [<8000807d>] (0x8000807d)
4------------[ cut here ]------------
4WARNING: CPU: 0 PID: 0 at kernel/time/clockevents.c:44
cev_delta2ns+0xcd/0xe4()
dModules linked in:
dCPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0+ #2
[<c0012315>] (unwind_backtrace+0x1/0x9c) from [<c000fbf1>]
(show_stack+0x11/0x14)
[<c000fbf1>] (show_stack+0x11/0x14) from [<c0478d5f>] (dump_stack+0x4f/0x64)
[<c0478d5f>] (dump_stack+0x4f/0x64) from [<c001c439>]
(warn_slowpath_common+0x51/0x70)
[<c001c439>] (warn_slowpath_common+0x51/0x70) from [<c001c46f>]
(warn_slowpath_null+0x17/0x1c)
[<c001c46f>] (warn_slowpath_null+0x17/0x1c) from [<c0058a91>]
(cev_delta2ns+0xcd/0xe4)
[<c0058a91>] (cev_delta2ns+0xcd/0xe4) from [<c0058f93>]
(clockevents_config+0x4b/0x60)
[<c0058f93>] (clockevents_config+0x4b/0x60) from [<c0058fbb>]
(clockevents_config_and_register+0x13/0x1c)
[<c0058fbb>] (clockevents_config_and_register+0x13/0x1c) from
[<c038f1db>] (arch_timer_setup+0x67/0x124)
[<c038f1db>] (arch_timer_setup+0x67/0x124) from [<c070848d>]
(arch_timer_init+0xf5/0x160)
[<c070848d>] (arch_timer_init+0xf5/0x160) from [<c0707d21>]
(clocksource_of_init+0x25/0x40)
[<c0707d21>] (clocksource_of_init+0x25/0x40) from [<c06e67bb>]
(start_kernel+0x187/0x310)
[<c06e67bb>] (start_kernel+0x187/0x310) from [<8000807d>] (0x8000807d)
4---[ end trace 15c15b4afa9eff8e ]---
6Architected cp15 timer(s) running at 0.00MHz (virt).
Division by zero in kernel.
dCPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W    3.13.0+ #2
[<c0012315>] (unwind_backtrace+0x1/0x9c) from [<c000fbf1>]
(show_stack+0x11/0x14)
[<c000fbf1>] (show_stack+0x11/0x14) from [<c0478d5f>] (dump_stack+0x4f/0x64)
[<c0478d5f>] (dump_stack+0x4f/0x64) from [<c025ee7f>] (Ldiv0_64+0x9/0x1a)
[<c025ee7f>] (Ldiv0_64+0x9/0x1a) from [<c0056625>]
(__clocksource_updatefreq_scale+0x35/0x158)
[<c0056625>] (__clocksource_updatefreq_scale+0x35/0x158) from
[<c0056757>] (__clocksource_register_scale+0xf/0x3c)
[<c0056757>] (__clocksource_register_scale+0xf/0x3c) from [<c070831f>]
(arch_timer_common_init+0xff/0x178)
[<c070831f>] (arch_timer_common_init+0xff/0x178) from [<c0707d21>]
(clocksource_of_init+0x25/0x40)
[<c0707d21>] (clocksource_of_init+0x25/0x40) from [<c06e67bb>]
(start_kernel+0x187/0x310)
[<c06e67bb>] (start_kernel+0x187/0x310) from [<8000807d>] (0x8000807d)
Division by zero in kernel.
dCPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W    3.13.0+ #2
[<c0012315>] (unwind_backtrace+0x1/0x9c) from [<c000fbf1>]
(show_stack+0x11/0x14)
[<c000fbf1>] (show_stack+0x11/0x14) from [<c0478d5f>] (dump_stack+0x4f/0x64)
[<c0478d5f>] (dump_stack+0x4f/0x64) from [<c025ee7f>] (Ldiv0_64+0x9/0x1a)
[<c025ee7f>] (Ldiv0_64+0x9/0x1a) from [<c00564d5>]
(clocks_calc_mult_shift+0x85/0xb8)
[<c00564d5>] (clocks_calc_mult_shift+0x85/0xb8) from [<c0056673>]
(__clocksource_updatefreq_scale+0x83/0x158)
[<c0056673>] (__clocksource_updatefreq_scale+0x83/0x158) from
[<c0056757>] (__clocksource_register_scale+0xf/0x3c)
[<c0056757>] (__clocksource_register_scale+0xf/0x3c) from [<c070831f>]
(arch_timer_common_init+0xff/0x178)
[<c070831f>] (arch_timer_common_init+0xff/0x178) from [<c0707d21>]
(clocksource_of_init+0x25/0x40)
[<c0707d21>] (clocksource_of_init+0x25/0x40) from [<c06e67bb>]
(start_kernel+0x187/0x310)
[<c06e67bb>] (start_kernel+0x187/0x310) from [<8000807d>] (0x8000807d)
Division by zero in kernel.
dCPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W    3.13.0+ #2
[<c0012315>] (unwind_backtrace+0x1/0x9c) from [<c000fbf1>]
(show_stack+0x11/0x14)
[<c000fbf1>] (show_stack+0x11/0x14) from [<c0478d5f>] (dump_stack+0x4f/0x64)
[<c0478d5f>] (dump_stack+0x4f/0x64) from [<c025ee7f>] (Ldiv0_64+0x9/0x1a)
[<c025ee7f>] (Ldiv0_64+0x9/0x1a) from [<c00564d5>]
(clocks_calc_mult_shift+0x85/0xb8)
[<c00564d5>] (clocks_calc_mult_shift+0x85/0xb8) from [<c06ef683>]
(sched_clock_register+0x7b/0x12c)
[<c06ef683>] (sched_clock_register+0x7b/0x12c) from [<c0708343>]
(arch_timer_common_init+0x123/0x178)
[<c0708343>] (arch_timer_common_init+0x123/0x178) from [<c0707d21>]
(clocksource_of_init+0x25/0x40)
[<c0707d21>] (clocksource_of_init+0x25/0x40) from [<c06e67bb>]
(start_kernel+0x187/0x310)
[<c06e67bb>] (start_kernel+0x187/0x310) from [<8000807d>] (0x8000807d)
6sched_clock: 56 bits at 0 Hz, resolution 0ns, wraps every 0ns

Thanks
- Suriyan






On Mon, Mar 17, 2014 at 10:39 AM, Suriyan Ramasami <suriyan.r@xxxxxxxxx> wrote:
> On Mon, Mar 17, 2014 at 6:52 AM, Julien Grall <julien.grall@xxxxxxxxxx> wrote:
>> On 03/17/2014 01:32 PM, Suriyan Ramasami wrote:
>>> On Mon, Mar 17, 2014 at 6:21 AM, Julien Grall <julien.grall@xxxxxxxxxx> 
>>> wrote:
>>>> Hello Suriyan,
>>>>
>>>> On 03/17/2014 01:17 PM, Suriyan Ramasami wrote:
>>>>> On Sun, Mar 16, 2014 at 1:31 PM, Julien Grall <julien.grall@xxxxxxxxxx> 
>>>>> wrote:
>>>>>> Can you post the modification you made? (at least where does the print 
>>>>>> come
>>>>>> from).
>>>>> The boot_count print is coming from a mod in xen/arch/arm/time.c in
>>>>> function init_xen_time()
>>>>> - printk("boot_count: %llu reread: %llu\n", boot_count,
>>>>> READ_SYSREG64(CNTPCT_EL0));
>>>>> just before the function returns. The timer is started up in u-boot.
>>>>> All the other printfs that you see with "Calling" is at the start of
>>>>> the fucntions - I was trying to figure out if xen was hanging - this
>>>>> is not the case though.
>>>>>
>>>>> I shall reply further on Ian's email where I do see dom0 hanging in
>>>>> calibrate_delay(). If I pass lpj=4464640 in the kernel parameters, it
>>>>> then loops in do_xor_speed() - specifically in the line while ((now =
>>>>> jiffies) == j), which points to jiffies not getting incremented at
>>>>> all!
>>>>
>>>> I remembered to have the same issue with Arndale a while ago. Which
>>>> kernel are you using? Can you give the dom0 log?
>>>>
>>> I am using linux kernel 3.13 which boots without xen. With xen, I do
>>
>> By 3.13, you mean https://github.com/hardkernel/linux.git branch
>> odroid-3.13.y-linaro, right?
> Yes.
>>
>>> not see any dom0 output (I even added the patch that you suggested in
>>> http://lists.xen.org/archives/html/xen-devel/2013-04/msg02387.html) -
>>> still no output from dom0.
>>
>>> I am pasting the linux output without xen, if that helps:
>>
>> Hmmm ... the exynos 5250 has it's own timer (samsung,exynos4210-mct)
>> which shouldn't be passthrough to dom0. I suspect that your issue come
>> from there.
>>
>> Did you add specific platform code for the board? Can you uncomment
>> #define DEBUG_DT in xen/arch/arm/domain_build.c and paste the log on
>> pastebin?
>>
>
> Xen code changes:
>
> 1. I have modified xen/include/asm-arm/platforms/exynos5.h This will
> be usefull when we kick the other CPUs.
> /*#define S5P_PA_SYSRAM   0x02020000*/
> #define S5P_PA_SYSRAM   0x0207301c
>
> 2. Also, xen/arch/arm/platform/exynos5.c
> static const char * const exynos5_dt_compat[] __initconst =
> {
>     "samsung,exynos5250",
>     "samsung,exynos5410",
>     NULL
> };
>
> 3. As per your suggestion -> #define DEBUG_DT in xen/arch/arm/domain_build.c
>
> In my previous emails I had mentioned the dom0 PC reflecting a hang in
> xor or calibrate_loop(), which was without the above patches (I had a
> xen compiled with the latest xen in git - 4.5 devel withouth the
> exynos5410 related patches, hence the mct got passed to dom0, and as
> you had earlier mentioned resulted in issues in dom0),
>
> With the patches that I have mentioned above in xen 4.4 stable, I see
> the dom0 PC in these functions:
>
> The dts was patched as follows:
> 1. Add entry for timer: (copied from Arndale - I know frequency is
> 24M, but I have no idea about the other values)
>         timer {
>                 compatible = "arm,cortex-a15-timer",
>                         "arm,armv7-timer";
>                 interrupts = <1 13 0xf08>,
>                         <1 14 0xf08>,
>                         <1 11 0xf08>,
>                         <1 10 0xf08>;
>                 clock-frequency = <24000000>;
>         };
>
> 2. Rename timer@101C0000 to mct@101C0000. It now looks like:
>         mct@101C0000 {
>                 compatible = "samsung,exynos4210-mct";
>                 reg = <0x101C0000 0xB00>;
>                  ... etc
>
> Without the above renaming (timer@101C0000 to mct@101C0000), linux was
> not booting on its own (without xen)
>
> 3. Comment out all the cpus except the first one. This is as per your
> suggestion previously.
>
> The dom0 kernel has the below XEN related configs: (enabled XEN and
> Virtualisation)
> [suriyan@Stealth linux-xu-3.13]$ grep XEN .config
> CONFIG_XEN_DOM0=y
> CONFIG_XEN=y
> CONFIG_XEN_BLKDEV_FRONTEND=y
> # CONFIG_XEN_BLKDEV_BACKEND is not set
> CONFIG_XEN_NETDEV_FRONTEND=y
> # CONFIG_XEN_NETDEV_BACKEND is not set
> CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
> CONFIG_HVC_XEN=y
> CONFIG_HVC_XEN_FRONTEND=y
> CONFIG_XEN_FBDEV_FRONTEND=y
> CONFIG_XEN_DEV_EVTCHN=y
> CONFIG_XEN_BACKEND=y
> CONFIG_XENFS=y
> CONFIG_XEN_COMPAT_XENFS=y
> CONFIG_XEN_SYS_HYPERVISOR=y
> CONFIG_XEN_XENBUS_FRONTEND=y
> CONFIG_XEN_GNTDEV=m
> CONFIG_XEN_GRANT_DEV_ALLOC=m
> CONFIG_SWIOTLB_XEN=y
> CONFIG_XEN_PRIVCMD=y
>
> 4. Changed kernel/printk/printk.c - printk() to call xen_raw_console_write()
> //      r = vprintk_emit(0, -1, NULL, 0, fmt, args);
>         r = vsnprintf(buf, sizeof(buf), fmt, args);
>         va_end(args);
>
>         xen_raw_console_write(buf);
>         return r;
>
>
> With the above, I am pasting the full logs of boot. I pressed '0' to
> get a few samples of what dom0 was doing.
> a. (XEN) PC:     c0053c88
> c0053bc4 T ktime_get
> c0053cc8 T pvclock_gtod_unregister_notifier
>
> b. (XEN) PC:     c00520e6
> c00520a8 T rcu_irq_exit
> c0052120 T rcu_irq_enter
>
> c. (XEN) PC:     c005555e
> c00554f4 T get_xtime_and_monotonic_and_sleep_offset
> c0055574 T ktime_get_update_offsets
>
> d. (XEN) PC:     c00472ce
> c0047278 T do_raw_spin_unlock
> c0047308 T do_raw_read_lock
>
> e. (XEN) PC:     c000843a
> c000840c t gic_handle_irq
> c0008464 T __exception_text_end
>
> Full logs in http://pastebin.com/kF2EV6KH
>
> Thanks so much!
> - Suriyan
>
>
>
>> Thanks,
>>
>> --
>> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.