[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [OSSTEST PATCH] jessie: Disable use of security.debian.org

We have about a 10% failure rate of a problem where the symptoms are
that the test box fails to get some thing from security.debian.org.

The apt-cacher-ng logs show the relevant test box's ip address
fetching the file that it is supposed to.  But, it is possible that
there are different timeouts, so that does not mean the problem is
inside the colo.

Fetching from other apt sources, notably the main Debian archive and
snapshot.d.o, do not seem to be affected.

Specifically, I searched the logs for the last 1000 host install steps,
and looked for the failures, with the following rune:

 select flight,job,logfile,started from (select *, (select val from runvars r 
where r.job=steps.job and r.flight=steps.flight and r.name='host')  from steps 
where testid like 'host-install%' and flight>130000 order by finished desc 
limit 1000) sub where status='broken';

I then used these runes to correlate that with the syslogs from the

 perl <~/t -ne 'use strict; s/^ *//; my ($flight,$job,$logf) = split / +\| +/; 
next unless $flight =~ m/^\d+$/; my $f = "$flight/$job/?.ts-syslog-server.log"; 
my @y = glob $f; print $_, "\n" for @y;' >~/u

 xargs <~/junk/u egrep -L 'Failed to fetch 
http://security\.debian\.org.*Connection failed'

The only logs which did not mention that error message were three
failed jobs on the same host, joubertin1, which seems not to be
rebooting reliably.

So I think this is a problem with the security.debian.org CDN.

For now, disable security updates entirely.  We don't really care
about the security patch status of test boxes anyway.  Hopefully this
will cause the system to become reliable again.

CC: Juergen Gross <jgross@xxxxxxxx>
Signed-off-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
 Osstest/Debian.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 17dfef54..d991bf48 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -932,7 +932,8 @@ END
     $preseed .= <<'END'
 d-i apt-setup/services-select multiselect updates
-        if $suite =~ m/jessie/ && $r{arch} eq 'arm64';
+    #   if $suite =~ m/jessie/;
+    # security.d.o CDN seems unreliable right now
     $preseed .= <<"END";

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.