Re: request for feedback on a Xen/Linux compatibility issue

[Jan Beulich <jbeulich@xxxxxxxx>]

On 06.01.2022 08:13, Juergen Gross wrote:
> On 06.01.22 01:40, Stefano Stabellini wrote:
>> Hi all,
>>
>> Today Xen dom0less guests are not "Xen aware": the hypervisor node
>> (compatible = "xen,xen") is missing from dom0less domUs device trees and
>> as a consequence Linux initializes as if Xen is not present. The reason
>> is that interfaces like grant table and xenstore (xenbus in Linux) don't
>> work correctly in a dom0less environment at the moment.
>>
>> The good news is that I have patches for Xen to implement PV drivers
>> support for dom0less guests. They also add the hypervisor node to device
>> tree for dom0less guests so that Linux can discover the presence of Xen
>> and related interfaces.
>>
>> When the Linux kernel is booting as dom0less kernel, it needs to delay
>> the xenbus initialization until the interface becomes ready. Attempts to
>> initialize xenbus straight away lead to failure, which is fine because
>> xenbus has never worked in Linux when running as dom0less guest up until
>> now. It is reasonable that a user needs a newer Linux to take advantage
>> of dom0less with PV drivers. So:
>>
>> - old Xen + old/new Linux -> Xen not detected in Linux
>> - new Xen + old Linux     -> xenbus fails to initialize in Linux
>> - new Xen + new Linux     -> dom0less PV drivers working in Linux
>>
>>
>> The problem is that Linux until recently couldn't deal with any errors
>> in xenbus initialization. Instead of returning error and continuing
>> without xenbus, Linux would crash at boot.
>>
>> I upstreamed two patches for Linux xenbus_probe to be able to deal with
>> initialization errors. With those two fixes, Linux can boot as a
>> dom0less kernel with the hypervisor node in device tree. The two fixes
>> got applied to master and were already backported to all the supported
>> Linux stable trees, so as of today:
>>
>> - dom0less with hypervisor node + Linux 5.16+           -> works
>> - dom0less with hypervisor node + stable Linux 5.10     -> works
>> - dom0less with hypervisor node + unpatched Linux 5.10  -> crashes
>>
>>
>> Is this good enough? Or for Xen/Linux compatibility we want to also be
>> able to boot vanilla unpatched Linux 5.10 as dom0less kernel? If so,
>> the simplest solution is to change compatible string for the hypervisor
>> node, so that old Linux wouldn't recognize Xen presence and wouldn't try
>> to initialize xenbus (so it wouldn't crash on failure). New Linux can of
>> course learn to recognize both the old and the new compatible strings.
>> (For instance it could be compatible = "xen,xen-v2".) I have prototyped
>> and tested this solution successfully but I am not convinced it is the
>> right way to go.
>>
>> Do you have any suggestion or feedback?
>>
>> The Linux crash on xenbus initialization failure is a Linux bug, not a
>> Xen issue. For this reason, I am tempted to say that we shouldn't change
>> compatible string to work-around a Linux bug, especially given that the
>> Linux stable trees are already all fixed.
> 
> What about adding an option to your Xen patches to omit the hypervisor
> node in the device tree? This would enable the user to have a mode
> compatible to today's behavior.

While this sounds nice at the first glance, this would need to be a per-
domain setting. Which wouldn't be straightforward to express via command
line option (don't know how feasible it would be to express such via other
means).

Jan