[Xen-devel] [PATCH v5 08/17] osstest: add a FreeBSD host install script

[Roger Pau Monne <roger.pau@xxxxxxxxxx>]

The installation is performed using the bsdinstall tool, which is part
of the FreeBSD base system. The installer image is setup with the
osstest ssh keys and sshd enabled by default, which allows the test
harness to just ssh into the box, create the install config file and
launch the scripted install.

Currently the installation is done with ZFS only, in stripe mode, and
a single disk.

In order to support the FreeBSD installer the setup_netboot_memdisk
helper is used, that allows setting the pxe boot of a host using a
memdisk.

The install script either picks the binary images from the output of a
previous FreeBSD buildjob (yet to be introduced), or from the folder
pointed by freebsd_distpath. This folder should contain the installer
image (install.img) and the compressed install sets (kernel.txz,
base.txz) together with the MANIFEST file that contains the checksums.

Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
---
Changes since v4:
 - Remove get_sets_path routine and xopts variable.
 - Call resource_shared_mark_ready when the host is ready to be used.

Changes since v3:
 - Use sha256file instead of the sha256sum clu.
 - Shift parameters passed to the lock and copy script, so that
   parameter 0 is ignored.
 - Copy to .tmp and then move the FreeBSD image.
 - Use a single-line find rune to cleanup old images.
 - Add some missing "set -e".
 - Don't check the output of target_cmd_output_root against undefined,
   that's never going to be true.
 - Split the setup_netboot_memdisk helper code into a pre-patch.

Changes since v2:
 - Use sysrc instead of echo to modify rc.conf.

Changes since v1:
 - Use pkg bootstrap instead of pkg update.
 - Use freebsdbuildjob instead of freebsd_buildjob.
 - add "append raw" to the memdisk command line.
 - Use with-lock-ex instead of flock.
 - Use system_checked and an array to hold the locked command to
   execute.
 - Use an array to store the list of possible disk devices.
 - Use target_putfilecontents_root_stash in order to upload the
   install script to the target.
 - Remove the usage of TftpFreeBSDBase.
 - Use target_tftp_prefix in order to get the prefix for the tftp
   target image path.
 - Add a missing linewrap.
 - Replace the split used to get the runvar paths with a regexp.
 - Place the install sets at /root/osstest_sets instead of
   /tmp/osstest_sets.
 - Pick the sets/installer image from the freebsdbuildjob
   path_freebsdist or from the runvar variable freebsd_distpath.
 - Add a comment about which FreeBSD specific inputs the script
   consumes.
---
 ts-freebsd-host-install | 256 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 256 insertions(+)
 create mode 100755 ts-freebsd-host-install

diff --git a/ts-freebsd-host-install b/ts-freebsd-host-install
new file mode 100755
index 00000000..321763b0
--- /dev/null
+++ b/ts-freebsd-host-install
@@ -0,0 +1,256 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# The FreeBSD installer script consumes either one of the following
+# runvars:
+#
+# freebsd_distpath: points to a folder accessible by the controller that
+# should contain the installer image and the compressed install sets,
+# together with the MANIFEST file.
+#
+# freebsdbuildjob: points to a previous FreeBSD build job, that has
+# produced an installer image and compressed sets. The only runvar used
+# form the job pointed to by freebsdbuildjob is the path_freebsddist one,
+# that points to the folder containing the files described above.
+#
+# The installer image must be named 'install.img', and the sets
+# 'kernel.txz', 'base.txz' and finally the 'MANIFEST' file that contains
+# the checksums.
+
+use strict qw(vars);
+use DBI;
+use POSIX;
+
+unshift @INC, qw(.);
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho= selecthost($whhost);
+exit 0 if $ho->{Flags}{'no-reinstall'};
+exit 0 if $ho->{SharedReady};
+
+our $timeout = 1000;
+
+our @sets = qw(base.txz kernel.txz);
+
+our $path_prefix = $r{"freebsd_distpath"} ||
+                   get_stashed("path_freebsddist", $r{"freebsdbuildjob"});
+
+sub create_ssh_overlay () {
+    my $url = create_webfile($ho, "ssh.tar", sub {
+        my ($fh) = @_;
+        contents_make_cpio($fh, 'ustar',  "$c{OverlayLocal}/etc/ssh/");
+    });
+
+    return $url;
+}
+
+sub setup_netboot_installer () {
+    my $image = "$path_prefix/install.img";
+    my $pxeimg = target_tftp_prefix($ho) . "--freebsd.img";
+    my $hash = sha256file($image, 16);
+    my $tftp_freebsd = "$ho->{Tftp}{Path}/$ho->{Tftp}{TmpDir}/freebsd-images/";
+    my $script = <<'END';
+basedir=$1
+imagepath=$2
+sharedpath=$3
+targetpath=$4
+cd $basedir
+mkdir -p `dirname $sharedpath`
+if [ ! -f $sharedpath ]; then
+    cp $imagepath $sharedpath.tmp
+    mv $sharedpath.tmp $sharedpath
+fi
+rm -f $targetpath
+ln $sharedpath $targetpath
+# Prune old images not used anymore
+find `dirname $sharedpath` -links 1 -ctime +7 -delete
+END
+    my @cmd = ( "with-lock-ex", "-w", "$tftp_freebsd/lock",
+                "bash", "-exc", "$script", "x",
+                "$tftp_freebsd", "$image", "by-hash/$hash.img",
+                "$ho->{Tftp}{Path}/$pxeimg" );
+
+    ensuredir($tftp_freebsd);
+    system_checked(@cmd);
+
+    # Setup the pxelinux config file
+    logm("Booting from installer image at $pxeimg");
+    setup_netboot_memdisk($ho, $pxeimg);
+}
+
+sub install () {
+    my $authkeys = authorized_keys();
+    my $knownhosts = known_hosts();
+    my $sshd_keys_url = create_ssh_overlay();
+    my @disk_names = qw(ada0 da0 ad0);
+    my $target_sets = "/root/osstest_sets";
+    my $disk;
+    my $nic;
+
+    target_cmd_root($ho, 'chsh -s /bin/sh', 10);
+
+    logm("Trying to find a disk to install to");
+    $disk = target_cmd_output_root($ho, <<END, 30);
+set -e
+for disk in @disk_names; do
+    if [ -c "/dev/\$disk" ]; then
+        echo \$disk
+        exit 0
+    fi
+done
+exit 1
+END
+    logm("Using $disk as destination disk device");
+
+    logm("Trying to figure out primary nic device name");
+    $nic = target_cmd_output_root($ho, <<END, 30);
+set -e
+nics=`ifconfig -l`
+for nic in \$nics; do
+    addr=`ifconfig \$nic inet|grep inet|awk {'print \$2'}`
+    if [ "\$addr" = "$ho->{Ip}" ]; then
+        echo \$nic
+        exit 0
+    fi
+done
+exit 1
+END
+    logm("Using $nic as primary network interface");
+
+    logm("Uploading the install sets to the system");
+    target_cmd_root($ho, <<END, 30);
+set -e
+mkdir -p $target_sets
+mount -o size=1G -t tmpfs tmpfs $target_sets
+END
+
+    foreach my $set (@sets, "MANIFEST") {
+        target_putfile_root($ho, 600, "$path_prefix/$set",
+                            "$target_sets/$set");
+    }
+
+    logm("Creating the installer script");
+    target_putfilecontents_root_stash($ho, 10, <<END, '~/installscript');
+set -a
+BSDINSTALL_DISTDIR="$target_sets"
+ZFSBOOT_DISKS="$disk"
+DISTRIBUTIONS="@sets"
+nonInteractive=1
+
+#!/bin/sh
+set -ex
+
+# Setup nic and sshd
+sysrc ifconfig_$nic=DHCP
+sysrc sshd_enable=YES
+
+# Use syncronious DHCP in case the server is slow
+sysrc synchronous_dhclient=YES
+
+# Disable sendmail
+sysrc sendmail_enable=NO
+sysrc sendmail_submit_enable=NO
+sysrc sendmail_outbound_enable=NO
+sysrc sendmail_msp_queue_enable=NO
+
+# Set proxy for the pkg manager
+mkdir -p /usr/local/etc/
+cat << ENDPKG >> /usr/local/etc/pkg.conf
+pkg_env: { http_proxy = $c{HttpProxy} }
+default_always_yes: true
+assume_always_yes: true
+ENDPKG
+
+# Bootstap the package manager
+export HTTP_PROXY=$c{HttpProxy}
+export ASSUME_ALWAYS_YES=yes
+pkg bootstrap
+
+# Allow root user login and setup ssh keys
+chsh -s /bin/sh root
+echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config
+mkdir -p /root/.ssh
+cat << ENDKEYS > /root/.ssh/authorized_keys
+$authkeys
+ENDKEYS
+cat << ENDHOSTS > /root/.ssh/known_hosts
+$knownhosts
+ENDHOSTS
+
+# Fetch host keys
+fetch $sshd_keys_url -o - | tar -xf - -C /etc/ssh/
+# Set correct permissions
+chown root:wheel /etc/ssh/ssh_host_*_key*
+chmod 0600 /etc/ssh/ssh_host_*_key
+chmod 0644 /etc/ssh/ssh_host_*_key.pub
+
+# Add a osstest user
+pw useradd osstest -m
+chsh -s /bin/sh osstest
+mkdir -p /home/osstest/.ssh
+cat << ENDKEYS > /home/osstest/.ssh/authorized_keys
+$authkeys
+ENDKEYS
+cat << ENDHOSTS > /home/osstest/.ssh/known_hosts
+$knownhosts
+ENDHOSTS
+
+# Setup serial console
+printf "%s" "-h -S$c{Baud}" >> /boot.config
+cat << ENDBOOT >> /boot/loader.conf
+boot_serial="YES"
+comconsole_speed="$c{Baud}"
+console="comconsole"
+boot_verbose="YES"
+beastie_disable="YES"
+ENDBOOT
+END
+
+    logm("Launch the installer");
+    target_cmd_root($ho, 'bsdinstall script installscript', 1200);
+
+    target_reboot($ho);
+
+    logm("Waiting for the host to boot");
+    await_tcp(get_timeout($ho,'reboot',$timeout), 5, $ho);
+
+    logm("FreeBSD installed succesfully");
+}
+
+# Switch off, setup PXE and switch on to the installer
+power_state($ho, 0);
+setup_netboot_installer();
+power_cycle_sleep($ho);
+power_state($ho, 1);
+
+# Wait for the host to finish booting
+logm("Waiting for the installer to boot");
+await_tcp(get_timeout($ho,'reboot',$timeout), 5, $ho);
+
+# Next boot will be from local disk
+setup_netboot_local($ho);
+
+# Proceed with the install
+install();
+
+resource_shared_mark_ready($ho, "build-freebsd-".
+                                sha256file("$path_prefix/install.img", 16));
-- 
2.11.0 (Apple Git-81)


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel