[Xen-devel] [OSSTEST PATCH] mg-debian-installer-update: produce deterministic output

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

Currently rerunning mg-debian-install-update when the external files
have changed still produces differences in the local files produced
during post-processing.

Avoid these differences by:

  - Using gzip -n, which avoids storing a timestamp in the gzip
    header (as well as the name, which we don't need).
  - Using pax -M norm, which normalises all timestamps (among other
    things, such as the owner, which we don't care about)
  - Using tar --mtime, with a reference within the dpkg-deb created
    hierarchy (which has timestamps from the package and is therefore
    dependent only on the downloaded package revision)

With this the results of two invocations of
mg-debian-installer-update(-all) are identical (assuming no changes to
the downloaded files) as demonstrated by runnign this quick hack:

        #!/bin/bash

        set -ex

        TMP=$HOME/tmp/mg-di
        rm -rf $TMP
        mkdir -p $TMP

        cat >$TMP/config <<EOF
        TftpPath $TMP
        TftpDiBase debian-installer
        EOF
        cat $TMP/config

        mkdir -p $TMP/debian-installer

        export OSSTEST_CONFIG=production-config:$TMP/config

        # ./mg-debian-installer-update wheezy armhf firmware-bnx2
        ./mg-debian-installer-update-all

        find $TMP/debian-installer -type f -print0 | xargs -0x md5sum > 
$TMP/SUMS.BASE

        #rm -rf $TMP/debian-installer/*
        mv $TMP/debian-installer $TMP/debian-installer.org
        mkdir -p $TMP/debian-installer

        # ./mg-debian-installer-update wheezy armhf firmware-bnx2
        ./mg-debian-installer-update-all
        md5sum --quiet -c $TMP/SUMS.BASE

        exit 0

Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
---
 mg-debian-installer-update | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 3ae50fb..5d12cf0 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -78,8 +78,8 @@ for p in $packages; do
         curl -s "$site/$pkgfile" >$p.deb
         rm -rf x
         dpkg-deb -x $p.deb x
-        cd x; pax -x sv4cpio -s '%lib%/lib%' -w lib >../cpio; cd ..
-        gzip -9f cpio
+        cd x; pax -x sv4cpio -s '%lib%/lib%' -w -M norm lib >../cpio; cd ..
+        gzip -9nf cpio
         mv cpio.gz $p.cpio.gz
         rm -rf x
 done
@@ -135,13 +135,14 @@ if [ $arch = armhf ]; then
                          -path \*/kernel/drivers/scsi/\* -o \
                          -path \*/kernel/drivers/usb/dwc3/\* -o \
                          -path \*/kernel/drivers/usb/host/\* \) \
-          |pax -x sv4cpio -s '%lib%/lib%' -d -w >../cpio; cd ..
-    gzip -9f cpio
+          |pax -x sv4cpio -s '%lib%/lib%' -d -w -M norm >../cpio; cd ..
+    gzip -9nf cpio
     mv cpio.gz armmp.cpio.gz
     rm -rf dtbs/
     mkdir dtbs/
     mv x/usr/lib/linux-image-*-armmp/*.dtb dtbs/
-    tar -caf dtbs.tar.gz dtbs
+    tar --mtime=./x/usr/lib -cf dtbs.tar dtbs
+    gzip -9nf dtbs.tar
     rm -rf x
 fi
 
-- 
2.1.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel