[XenPPC] [PATCH] Flush the ERAT early for secondary CPUs

[Amos Waterland <apw@xxxxxxxxxx>]

Flush the ERAT very early on secondary processors.

Because Jimi has expressed skepticism about the need to do this, we
provide the following empirical and statistical arguments.  

In the results presented below, we observe 897 passes out of 900 trials
with vanilla Xen/PPC plus the SMP/IPI patch.  Note that we have observed
the "Synchronizing timebase hang" with pure vanilla Xen/PPC, but we test
here with the SMP/IPI patch because we observe the hang to occcur more
often with it.  With this patch applied, we observe 2323 passes out of
2323 trials.  In all cases, we use dom0 making it to a bash prompt as
the success criteria.  Since we have empirical evidence that the hang
occurs randomly, we declare the following:

 p = probability of success = .997 (897 / 900)
 q = probability of failure = .003 (1.0 - .997)
 n = number of trials       = 2323
 X = number of successes    = 2323

Applying these to the binomial probability formula, we get:

 P(2323) = 2323! / ((2323 - 2323)! * 2323!) * .997**2323 * .003**(2323-2323)
         = .0009307922

So we conclude that the probability that our trials with this patch
applied achieved exactly 2323 successes because of chance alone is .0009.

---

Reliability results for Xen with the SMP/IPI patch plus this ERAT flush
patch applied:

 changeset   : 25c51961bd3f+8c8eb
 machines    : kpblade1 cso103 cso102 kpblade7 cso98 cso99
 fail        : 0
 transient   : 6
 pass        : 2323
 total       : 2329
 reliability : 100%

Reliability results for Xen with the SMP/IPI patch applied:

 changeset   : 25c51961bd3f+00000
 machines    : kpblade1 cso103 cso102 kpblade7 cso98 cso99
 fail        : 3
 transient   : 4
 pass        : 897
 total       : 904
 reliability : 99%

 25c51961bd3f+00000/kpblade7/xen.log.2006-11-09.112408
 =====================================================
 (XEN) Synchronizing timebase

 25c51961bd3f+00000/cso102/xen.log.2006-11-09.114634
 ===================================================
 (XEN) Synchronizing timebase

 25c51961bd3f+00000/cso102/xen.log.2006-11-09.133343
 ===================================================
 spinning up secondary processor #3: ping = 0xffffffff:

Note that SLOF reverted to PHYP four times in the 904 runs without this
patch applied, and never in the 2329 runs with this patch applied.

Signed-off-by: Amos Waterland <apw@xxxxxxxxxx>

 exceptions.S |    4 ++++
 1 file changed, 4 insertions(+)

diff -r 25c51961bd3f xen/arch/powerpc/powerpc64/exceptions.S
--- a/xen/arch/powerpc/powerpc64/exceptions.S   Thu Oct 26 20:58:55 2006 -0400
+++ b/xen/arch/powerpc/powerpc64/exceptions.S   Wed Nov 08 22:10:06 2006 -0500
@@ -564,6 +564,10 @@ _GLOBAL(sleep)
  */    
     .globl spin_start
 spin_start:
+    /* We discovered by experiment that the ERAT must be flushed early.  */
+    isync
+    slbia
+    isync
 
     /* Do a cache flush for our text, in case the loader didn't */
     LOADADDR(r9, _start)

_______________________________________________
Xen-ppc-devel mailing list
Xen-ppc-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ppc-devel