Re: [Xen-devel] [PATCH v1 5/7] ts-livepatch: Initial test-cases.

[Konrad Rzeszutek Wilk <konrad@xxxxxxxxxx>]

On Thu, Nov 17, 2016 at 12:21:58PM +0000, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("[PATCH v1 5/7] ts-livepatch: Initial 
> test-cases."):
> > There are 32 test-cases in there. Before we run
> > any of them we verify that the payload files are present
> > in /usr/lib/debug.
> > 
> > If so we go through all of the test-cases.

Thank you!

Let me answer only to some as I am fixing the rest per your comments:
> 
> Is it ever possible to continue with the rest of the livepatch tests
> after one of these command invocations has failed, or does it leave
> the system in an undefined state ?  If it _is_ possible then it would

If the invocations have failed that could mean: 
 - We don't have livepatching enabled (somebody swapped the hypervisor?)
 - The livepatching did not work right and we have code in the
   hypervisor that patched something else. Undefined results.
 - It may be that the test-case failed to load due to dependencies
   issues - and while that means the system can recover - the test
   should fail immediately.

> be possible to provide per-step results to osstest, but that's only
> valuable if this would tell us something meaningfully interesting in
> terms of regressions - eg if we might tolerate a regression of one
> step but still care about the others.

At this stage I believe all of these test-cases should work. If they
don't we got big problems.
..
> > +   {cmd => "xen-livepatch revert xen_hello_world", rc => 256},
> 
> libxl exit statuses are not very good and should not be relied on,
> really.  Instead, you should arrange to:

But this is not libxl. The error values are very much defined
by xen-livepatch.

>   * Run the command with LC_MESSAGES=C

Aye.
>   * Fail the test if the exit status is zero

But some of the test-cases are suppose to return 0 - as the program
executed correctly.

>   * Collect its stderr output (by appending 2>&1)
>   * Match the stderr output against a regexp in the perl program

Right.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel