Re: [Xen-devel] ARM: EXYNOS 5410 - DOM0 not being scheduled

[Suriyan Ramasami <suriyan.r@xxxxxxxxx>]

On Mon, Mar 17, 2014 at 6:52 AM, Julien Grall <julien.grall@xxxxxxxxxx> wrote:
> On 03/17/2014 01:32 PM, Suriyan Ramasami wrote:
>> On Mon, Mar 17, 2014 at 6:21 AM, Julien Grall <julien.grall@xxxxxxxxxx> 
>> wrote:
>>> Hello Suriyan,
>>>
>>> On 03/17/2014 01:17 PM, Suriyan Ramasami wrote:
>>>> On Sun, Mar 16, 2014 at 1:31 PM, Julien Grall <julien.grall@xxxxxxxxxx> 
>>>> wrote:
>>>>> Can you post the modification you made? (at least where does the print 
>>>>> come
>>>>> from).
>>>> The boot_count print is coming from a mod in xen/arch/arm/time.c in
>>>> function init_xen_time()
>>>> - printk("boot_count: %llu reread: %llu\n", boot_count,
>>>> READ_SYSREG64(CNTPCT_EL0));
>>>> just before the function returns. The timer is started up in u-boot.
>>>> All the other printfs that you see with "Calling" is at the start of
>>>> the fucntions - I was trying to figure out if xen was hanging - this
>>>> is not the case though.
>>>>
>>>> I shall reply further on Ian's email where I do see dom0 hanging in
>>>> calibrate_delay(). If I pass lpj=4464640 in the kernel parameters, it
>>>> then loops in do_xor_speed() - specifically in the line while ((now =
>>>> jiffies) == j), which points to jiffies not getting incremented at
>>>> all!
>>>
>>> I remembered to have the same issue with Arndale a while ago. Which
>>> kernel are you using? Can you give the dom0 log?
>>>
>> I am using linux kernel 3.13 which boots without xen. With xen, I do
>
> By 3.13, you mean https://github.com/hardkernel/linux.git branch
> odroid-3.13.y-linaro, right?
Yes.
>
>> not see any dom0 output (I even added the patch that you suggested in
>> http://lists.xen.org/archives/html/xen-devel/2013-04/msg02387.html) -
>> still no output from dom0.
>
>> I am pasting the linux output without xen, if that helps:
>
> Hmmm ... the exynos 5250 has it's own timer (samsung,exynos4210-mct)
> which shouldn't be passthrough to dom0. I suspect that your issue come
> from there.
>
> Did you add specific platform code for the board? Can you uncomment
> #define DEBUG_DT in xen/arch/arm/domain_build.c and paste the log on
> pastebin?
>

Xen code changes:

1. I have modified xen/include/asm-arm/platforms/exynos5.h This will
be usefull when we kick the other CPUs.
/*#define S5P_PA_SYSRAM   0x02020000*/
#define S5P_PA_SYSRAM   0x0207301c

2. Also, xen/arch/arm/platform/exynos5.c
static const char * const exynos5_dt_compat[] __initconst =
{
    "samsung,exynos5250",
    "samsung,exynos5410",
    NULL
};

3. As per your suggestion -> #define DEBUG_DT in xen/arch/arm/domain_build.c

In my previous emails I had mentioned the dom0 PC reflecting a hang in
xor or calibrate_loop(), which was without the above patches (I had a
xen compiled with the latest xen in git - 4.5 devel withouth the
exynos5410 related patches, hence the mct got passed to dom0, and as
you had earlier mentioned resulted in issues in dom0),

With the patches that I have mentioned above in xen 4.4 stable, I see
the dom0 PC in these functions:

The dts was patched as follows:
1. Add entry for timer: (copied from Arndale - I know frequency is
24M, but I have no idea about the other values)
        timer {
                compatible = "arm,cortex-a15-timer",
                        "arm,armv7-timer";
                interrupts = <1 13 0xf08>,
                        <1 14 0xf08>,
                        <1 11 0xf08>,
                        <1 10 0xf08>;
                clock-frequency = <24000000>;
        };

2. Rename timer@101C0000 to mct@101C0000. It now looks like:
        mct@101C0000 {
                compatible = "samsung,exynos4210-mct";
                reg = <0x101C0000 0xB00>;
                 ... etc

Without the above renaming (timer@101C0000 to mct@101C0000), linux was
not booting on its own (without xen)

3. Comment out all the cpus except the first one. This is as per your
suggestion previously.

The dom0 kernel has the below XEN related configs: (enabled XEN and
Virtualisation)
[suriyan@Stealth linux-xu-3.13]$ grep XEN .config
CONFIG_XEN_DOM0=y
CONFIG_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=y
# CONFIG_XEN_BLKDEV_BACKEND is not set
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_BACKEND is not set
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
CONFIG_XEN_FBDEV_FRONTEND=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m
CONFIG_XEN_GRANT_DEV_ALLOC=m
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_PRIVCMD=y

4. Changed kernel/printk/printk.c - printk() to call xen_raw_console_write()
//      r = vprintk_emit(0, -1, NULL, 0, fmt, args);
        r = vsnprintf(buf, sizeof(buf), fmt, args);
        va_end(args);

        xen_raw_console_write(buf);
        return r;


With the above, I am pasting the full logs of boot. I pressed '0' to
get a few samples of what dom0 was doing.
a. (XEN) PC:     c0053c88
c0053bc4 T ktime_get
c0053cc8 T pvclock_gtod_unregister_notifier

b. (XEN) PC:     c00520e6
c00520a8 T rcu_irq_exit
c0052120 T rcu_irq_enter

c. (XEN) PC:     c005555e
c00554f4 T get_xtime_and_monotonic_and_sleep_offset
c0055574 T ktime_get_update_offsets

d. (XEN) PC:     c00472ce
c0047278 T do_raw_spin_unlock
c0047308 T do_raw_read_lock

e. (XEN) PC:     c000843a
c000840c t gic_handle_irq
c0008464 T __exception_text_end

Full logs in http://pastebin.com/kF2EV6KH

Thanks so much!
- Suriyan



> Thanks,
>
> --
> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel