[Xen-devel] Need some advice on how to debug a boot hang in console_lock() in radeon driver loading

["Zytaruk, Kelly" <Kelly.Zytaruk@xxxxxxx>]

Hi,

I recently upgraded from (Debian 7) linux 3.2.0 to 3.13.4 and I am having 
difficulties booting.
I have multiple versions on my machine; 3.2.0, 3.4.9, 3.13.4 and 3.13.5.  
3.2.0 and 3.4.9 have no problem but both 3.13.4 and 3.13.5 hang during boot. 

My system is hanging during the loading of the radeon driver.
Specifically, the call to do_register_framebuffer() is calling console_lock().
Within console_lock() it is calling down(&sem).  The hang is somewhere within 
the down() call as it never returns from down().

I have put printk() calls in console_lock() as shown below (and elsewhere in 
the radeon driver);

void console_lock(void)
{
        printk (KERN_ERR "console_lock() - console_locked = %d, sem count = 
%d\n", console_locked, console_sem.count);
        might_sleep();
        printk (KERN_ERR "console_lock() - call down()\n");
        down(&console_sem);
        if (console_suspended)
        {
                printk (KERN_ERR "console_lock() - console is suspended\n");
                return;
        }
        console_locked = 1;
        console_may_schedule = 1;
        printk (KERN_ERR "console_lock() - acquire the mutex\n");
        mutex_acquire(&console_lock_dep_map, 0, 0, _RET_IP_);
}

The output to a serial port during boot is;

. . . 
[    5.313569] console_lock() - console_locked = 0, sem count = 1
[    5.319930] console_lock() - call down()
[    5.325371] console_lock() - acquire the mutex
[    5.330140] console_lock() - console_locked = 0, sem count = 1
[    5.337362] console_lock() - call down()
[    5.341393] console_lock() - acquire the mutex
[ ok ] Starting the hotplug events dispatcher: udevd.
[ ok ] Synthesizing the initial hotplug events...done.
[....] Waiting for /dev to be fully populated...
[    6.636204] __pci_register_driver (i2c_piix4, piix4_smbus)
[    6.642085] __pci_register_driver (k10temp, k10temp)
[    6.656914] __pci_register_driver (ehci_pci, ehci-pci)
[    6.715643] __pci_register_driver (ohci_pci, ohci-pci)
[    6.922953] __pci_register_driver (amd64_edac_mod, amd64_edac)
[    7.815481] __pci_register_driver (snd_hda_intel, snd_hda_intel)
[    8.066608] Enter radeon_init(void)
[    8.070231] Exitting radeon_init(void),call drm_pci_init() on the way out
[    8.077173] Enter drm_pci_init()
[    8.080503] This is a Render device
[    8.084096] __pci_register_driver (drm, radeon)
[    8.089331] Enter radeon_driver_load_kms()
[    8.093585] Enter radeon_device_init()
[    8.136403] radeon 0000:00:01.0: radeon_uvd: Can't load firmware 
"radeon/BONAIRE_uvd.bin"
[    8.156979] Try to load uCode for KAVERI
 [    8.185785] request_firmware (radeon/KAVERI_pfp.bin) returns 0
[    9.208145] Exit radeon_device_init() with success
[    9.213105] radeon_driver_load_kms - call radeon_modeset_init()
[    9.219188] radeon_modeset_init() - call drm_mode_config_init()
[    9.225248] radeon_modeset_init() - max_width=16384, max_height=16384
[    9.231836] radeon_modeset_init() - call radeon_modeset_create_props()
[    9.238519] radeon_modeset_init() - call radeon_i2c_init()
[    9.244271] radeon_modeset_init() - call radeon_setup_enc_conn()
[    9.251532] radeon_modeset_init() - call radeon_atom_encoder_init()
[    9.279816] radeon_modeset_init() - call radeon_atom_disp_eng_pll_init()
[    9.286779] radeon_modeset_init() - call radeon_hpd_init()
[    9.292413] radeon_modeset_init() - call radeon_afmt_init()
[    9.298132] radeon_modeset_init() - call radeon_pm_init()
[    9.303728] radeon_modeset_init() - call radeon_fbdev_init()
[    9.320230] Exit: drm_helper_disable_unused_functions()
[    9.325607] drm_fb_helper_initial_config() call 
drm_fb_helper_parse_command_line()
[    9.333339] drm_fb_helper_initial_config() call 
drm_fb_helper_probe_connector_modes()
[    9.417229] drm_fb_helper_initial_config() call drm_setup_crtcs()
[    9.423527] drm_fb_helper_initial_config() call 
drm_fb_helper_single_fb_probe() on the way out
[    9.432316] Enter: drm_fb_helper_single_fb_probe()
[    9.437225] drm_fb_helper_single_fb_probe() - 3 connectors
[    9.442832] drm_fb_helper_single_fb_probe() call probing function
[    9.450542] drm_fb_helper_single_fb_probe() probing function returns 0
[    9.457273] drm_fb_helper_single_fb_probe() call register_framebuffer()
[    9.464026] Enter: register_framebuffer()
[    9.468047] Enter: do_register_framebuffer() - call fb_check_foreignness()
[    9.475102] do_register_framebuffer() call 
do_remove_conflicting_framebuffers()
[    9.482566] do_register_framebuffer() call device_create()
[    9.488250] do_register_framebuffer() call fb_init_device()
[    9.493959] do_register_framebuffer() call fb_var_to_videomode(true)
[    9.504827] do_register_framebuffer() call fb_add_videomode()
[    9.510738] do_register_framebuffer() call console_lock()
[    9.516257] console_lock() - console_locked = 0, sem count = 1
[    9.522220] console_lock() - call down()

The last line above is the last thing that I see from the serial port.
It took me quite a while to get this far with my investigation and now I am 
stuck on how to debug into the semaphore, printk calls don't work anymore.  I 
tried putting a printk into down() and it hung very early in the boot.

Any suggestions in how to proceed or what tools I need to continue would be 
appreciated, keeping in mind that my Linux background is still on the learning 
curve ( I started learning Linux about 6 months ago).  I have a serial port 
available on the system and if needed I could probably dig up a PCIe 1394 
firewire card as well.

I have two machines, 3.13.4 fails on one but works on the other.  The failing 
machine is a Kaveri system, the passing machine is a Tahiti on Intel, so I 
suspect something Kaveri specific but I don't see how the semaphore could be 
related to Kaveri.

Thanks,
Kelly




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel