|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [PATCH RESEND 2/2] gitlab-ci: add an ARM32 qemu-based smoke test
+Anthony, Bertrand
Apologies, this feel through the cracks!
Also considering the recent arm32 xen breakage, which could have been
caught by gitlab-ci before commit, I am keen on getting this short
series in. Better late than never :-)
I forward-ported the series to staging with minimal changes and it still
works! I'll send out an update shortly to xen-devel. I am also replying
here because there is an interesting open question about Debian Jessie
below.
One note: in case you didn't notice, we already have a couple of arm64
and one x86 test in gitlab-ci using QEMU and they all follow the same
pattern, with similar scripts to the one added here, see for instance
automation/scripts/qemu-smoke-arm64.sh.
On Wed, 21 Jul 2021, Julien Grall wrote:
> Hi Stefano,
>
> On 14/07/2021 01:48, Stefano Stabellini wrote:
> > Add a minimal ARM32 smoke test based on qemu-system-arm, as provided by
> > the test-artifacts qemu container. The minimal test simply boots Xen
> > (built from previous build stages) and Dom0. The test is fetching the
> > Dom0 kernel and initrd from Debian Jessie: they work just fine and this
> > way we don't have to maintain a build for them too.
>
> Debian Jessie is not in LTS anymore. I am entirely sure how long this will
> still be available (older release seems to have disappeared). I would suggest
> to use Bullseye or at least Buster.
This is a good point and unfortunately I don't have a good answer. I
tried to use Bullseye and Buster kernels but neither of them work. I
don't know why. I tried to use a couple of kernels from Alpine Linux,
but they also don't work.
Curious I started bisecting kernels using "make multi_v7_defconfig +
CONFIG_XEN" and discovered that something broke with the 4.9 release.
Up until 4.8 they work fine in the default config. I have not
investigated further yet.
We still have a few months of support in Debian Jessie. My suggestion is
to go with Debian Jessie for now in this series. Then depending on the
result of the kernel investigation we can decide what to do.
I could also move the fetching of the kernel and initrd from
qemu-smoke-arm32.sh to the containiner build so that if Debian Jessie
goes offline it wouldn't affect us immediately. Not required as of now,
but could be good to do.
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > ---
> > automation/gitlab-ci/test.yaml | 23 ++++++++
> > automation/scripts/qemu-smoke-arm32.sh | 72 ++++++++++++++++++++++++++
> > 2 files changed, 95 insertions(+)
> > create mode 100755 automation/scripts/qemu-smoke-arm32.sh
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index d3b670b156..7f942dd64b 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -71,6 +71,29 @@ qemu-smoke-arm64-gcc:
> > - /^coverity-tested\/.*/
> > - /^stable-.*/
> > +qemu-smoke-arm32-gcc:
> > + stage: test
> > + image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > + variables:
> > + CONTAINER: debian:unstable-arm64v8
> > + script:
> > + - ./automation/scripts/qemu-smoke-arm32.sh 2>&1 | tee
> > qemu-smoke-arm32.log
> > + dependencies:
> > + - debian-unstable-gcc-arm32
> > + - qemu-system-aarch64-6.0.0-arm32-export
> > + artifacts:
> > + paths:
> > + - smoke.serial
> > + - '*.log'
> > + when: always
> > + tags:
> > + - arm64
> > + except:
> > + - master
> > + - smoke
> > + - /^coverity-tested\/.*/
> > + - /^stable-.*/
> > +
> > qemu-smoke-x86-64-gcc:
> > stage: test
> > image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > diff --git a/automation/scripts/qemu-smoke-arm32.sh
> > b/automation/scripts/qemu-smoke-arm32.sh
> > new file mode 100755
> > index 0000000000..4126be2042
> > --- /dev/null
> > +++ b/automation/scripts/qemu-smoke-arm32.sh
> > @@ -0,0 +1,72 @@
> > +#!/bin/bash
> > +
> > +set -ex
> > +
> > +# Install QEMU
>
> The bits below doesn't seem to install QEMU. Do you mean "prepare the image"?
I updated the comment
> > +export DEBIAN_FRONTENT=noninteractive
> > +apt-get -qy update
> > +apt-get -qy install --no-install-recommends device-tree-compiler \
> > + curl
> > +
> > +cd binaries
> > +curl -fsSLO
> > http://http.us.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/vmlinuz
> > +curl -fsSLO
> > http://http.us.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/initrd.gz
>
> OOI, does this mean all our instances are running in the US?
Yes. However, I wanted to replace this link with one of those links that
serves you the best mirror available at the moment. But I couldn't find
any. So I left it as is.
> > +
> > +kernel=`stat -L --printf="%s" vmlinuz`
> > +initrd=`stat -L --printf="%s" initrd.gz`
> > +
>
> NIT: I would suggest to add a comment similar to:
>
> # For Xen, we need a couple of more node. Dump the DT from QEMU and add them
I updated the comment
> > +# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
>
> Is this a problem with the package?
Yes it is a problem with both QEMU (which shouldn't be looking for
efi-virtio.rom when calling dumpdtb) and also the package (that doesn't
provide it).
We have the same workaround in all the qemu scripts under automation.
> > +curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> > +./qemu-system-arm \
> > + -machine virt-6.0 \
> > + -machine virtualization=true \
> > + -smp 4 \
> > + -m 2048 \
> > + -serial stdio \
> > + -monitor none \
> > + -display none \
> > + -machine dumpdtb=virt.dtb
> > +
> > +dtc -I dtb -O dts virt.dtb > virt.dts
>
> IIRC, Alex was working on add multiboot support in QEMU. Do you know what's
> the state?
I don't know the latest
> > +
> > +cat >> virt.dts << EOF
> > +/ {
> > + chosen {
> > + #address-cells = <0x2>;
> > + #size-cells = <0x2>;
> > + stdout-path = "/pl011@9000000";
> > + xen,xen-bootargs = "console=dtuart dtuart=/pl011@9000000
> > dom0_mem=1G dom0_max_vcpus=1 bootscrub=0";
>
> Given that QEMU will expose 4 CPUs, any reason to limit dom0 to 1?
Good idea, I did that
> > + xen,dom0-bootargs = "console=tty0 console=hvc0 earlyprintk
> > clk_ignore_unused root=/dev/ram0 rdinit=/bin/sh init=/bin/sh";
> > + dom0 {
> > + compatible = "xen,linux-zimage",
> > "xen,multiboot-module";
> > + reg = <0x0 0x1000000 0x0 $kernel>;
> > + };
> > + dom0-ramdisk {
> > + compatible = "xen,linux-initrd",
> > "xen,multiboot-module";
> > + reg = <0x0 0x3200000 0x0 $initrd>;
> > + };
> > + };
> > +};
> > +EOF
> > +dtc -I dts -O dtb virt.dts > virt.dtb
> > +
> > +rm -f smoke.serial
> > +set +e
> > +timeout -k 1 240 \
> > +./qemu-system-arm \
> > + -machine virt-6.0 \
> > + -machine virtualization=true \
> > + -smp 4 \
> > + -m 2048 \
>
> Given seem to be a bit wasteful (and will definitely slow down Xen boot). As
> this is a smoke, can we consider to drop to 512?
Yes, also good idea. I dropped dom0_mem to 512 and here -m 1024. Less
than that, it leads to a boot error (I haven't investigated).
> > + -serial stdio \
> > + -monitor none \
> > + -display none \
> > + -dtb virt.dtb \
> > + -no-reboot \
> > + -kernel ./xen \
> > + -device loader,file=./vmlinuz,addr=0x1000000 \
> > + -device loader,file=./initrd.gz,addr=0x3200000 |& tee smoke.serial
> > +
> > +set -e
> > +(grep -q "^BusyBox" smoke.serial) || exit 1
>
> So this look for "BusyBox" anywhere in the log. What if there is a crash just
> after "BusyBox"? Is there something different we can grep?
I am not sure, there isn't much after "BusyBox" because the initrd is
basically empty (appended below the last few lines for your reference).
Also keeping in mind that all the QEMU tests today check for "BusyBox",
I would leave it as is.
---
BusyBox v1.22.1 (Debian 1:1.22.0-9+deb8u1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
/bin/sh: can't access tty; job control turned off
~ #
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |