[PATCH 5/6] automation/qtb: add QTB framework README

[Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>]

Document the qtb riscv64 test framework in a README.

It covers:
  - the core concepts (machine, test type, test) and how they map to files
  - the source files layout
  - the CLI: `qemu_smoke_riscv64.py <type> <command>`, with "console-test"
    as the type
  - the config files: the machine catalog and a type's own `<type>.yaml`
  - the Jinja2 device-tree templates under dts/
  - how to add a test (config-only) and how to add a new test type.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>
---
 automation/scripts/qtb/riscv/README.md | 182 +++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
 create mode 100644 automation/scripts/qtb/riscv/README.md

diff --git a/automation/scripts/qtb/riscv/README.md 
b/automation/scripts/qtb/riscv/README.md
new file mode 100644
index 0000000000..ffcb027bbc
--- /dev/null
+++ b/automation/scripts/qtb/riscv/README.md
@@ -0,0 +1,182 @@
+qtb riscv64 test framework
+==========================
+
+A small framework that boots Xen under QEMU on riscv64 and drives it to a
+pass/fail verdict automatically from its console output.
+
+It is built on QEMU's qtb (QEMU Test Bench) Python package, which gives
+programmatic control of a QEMU process over QMP and qtest, plus access to the
+consoles.
+
+What it does
+------------
+
+1. Reads a machine description (what to boot: cpus, Xen command line) and a
+   test description (what to assert).
+2. Generates the host device tree.
+3. Launches QEMU with Xen and the firmware wired in.
+4. Reads the console and checks what Xen printed.
+
+Core concepts
+-------------
+
+- Machine: test-agnostic description of what to boot. Reusable across test
+  types. `config.yaml` -> `MachineConfig`.
+- Test type: a `RiscvQtbTest` subclass implementing the logic of a kind of test
+  (e.g. `console-test`). Identified by `type_id`. See `console_test/`.
+- Test: one named, runnable instance of a type: a machine plus the type's
+  parameters. Lives in the type's `<type>.yaml`.
+
+A test type owns a config file describing its tests, each test names a machine
+from the shared catalog (`config.yaml`) and layers its own parameters on top.
+
+Layout
+------
+
+```
+qemu_smoke_riscv64.py        CLI entry point (<type> run | list)
+
+qtb/riscv/                   This framework
+  __init__.py                Public API
+  qtb_test.py                RiscvQtbTest ABC every test type derives from
+  config.py                  Machine catalog parser -> MachineConfig
+  xen_dt.py                  Generates the host device tree from its Jinja2 
template
+  dt.py                      Compile .dts -> .dtb with dtc
+  paths.py                   Path resolution (pkg-relative)
+  machine.py                 RiscvTestMachine: assembles the QEMU command line
+
+  config.yaml                The machine catalog (shared across test types)
+  dts/                       Jinja2 device-tree templates (host, common)
+
+  console_test/              The console-test type
+    __init__.py
+    console_test.py          ConsoleTest implementation
+    console-test.yaml        Its tests
+
+  unit/                      pytest unit tests of the framework logic itself
+```
+
+How a type is selected
+----------------------
+
+The test type is the first positional argument (`qemu_smoke_riscv64.py 
console-test run
+...`). The CLI builds one subcommand per entry of `TEST_TYPES` (`__init__.py`),
+named after the type's `type_id`.
+
+Each type declares the `config_file` it reads its tests from.
+
+Prerequisites
+-------------
+
+- `qemu.qtb`, QEMU's Python package (`python/` in the QEMU tree)
+- `jinja2`, `pyyaml`, `pexpect`
+- `dtc` (device-tree-compiler)
+- the binaries a machine boots: `qemu-system-riscv64`, the firmware
+  (OpenSBI) and `xen`
+
+CLI usage
+---------
+
+Run from `automation/scripts/`, or give the full path from the Xen tree root
+(`./automation/scripts/qemu_smoke_riscv64.py ...`), which is what CI does.
+
+```
+# List every test the type defines in its config:
+./qemu_smoke_riscv64.py console-test list
+
+# Run one test (drives it to PASS/FAIL, exit 0/1):
+./qemu_smoke_riscv64.py console-test run 
dom0less-1smp-0domu-1vcpu-aplic-imsic-null \
+    --log-dir qtb-logs
+```
+
+`--log-dir` (run only) collects the QEMU process log, the qtest log, and each
+console as `con<N>.log`: `con0.log` is Xen's own console, the only one wired
+up today. Omit it to write no logs. `-v/--verbose` raises the
+log level to debug.
+
+Config files
+------------
+
+`config.yaml` is the machine catalog. `binaries:` are build artifacts resolved
+under `binaries/` (overridable with `$QTB_BINARIES_DIR`); absolute paths pass
+through.
+
+Machine entries omit any optional field left at its default.
+Here are the parameters:
+
+- `pcpu` (required): host physical cpus.
+- `mmu_type` (default `sv48`): Xen host MMU type.
+- `xen_bootargs` (default `""`): Xen command line.
+
+`<type>.yaml` describes the tests of that type.
+
+`console-test`
+--------------
+
+A test names a machine and maps a console index to the string(s) expected on
+that console: index 0 is Xen's own console (`con0`, logged as `con0.log`).
+
+```
+machine_catalog: config.yaml    # the catalog to resolve machine names against
+tests:
+  dom0less-1smp-0domu-1vcpu-aplic-imsic-null:
+    machine: dom0less-1smp-0domu-1vcpu-aplic-imsic-null   # a name in 
config.yaml
+    expect:
+      0: [All set up]           # Xen itself must print "All set up"
+```
+
+Logic, per console:
+
+1. read the console
+2. wait for each expected string in turn, in the order listed
+3. bound each wait by `timeout` seconds, retrying a timed-out wait up to
+   `attempts` times
+
+The map itself must not be empty, otherwise the test would pass without
+asserting anything.
+
+Device trees (dts/)
+-------------------
+
+Jinja2 template, generated per machine and compiled with dtc:
+
+- `qemu-host.dts.j2` - the Xen host tree: the hart count, the host MMU type and
+  the Xen command line.
+
+Adding a test
+-------------
+
+To add a test to an existing type (e.g. `console-test`):
+
+1. Pick a machine from `config.yaml`, or add a new one under `machines:` (set
+   `pcpu`, and any optional field that differs from its default — see the
+   field list above).
+2. Add a test entry under `tests:` in the type's `<type>.yaml`, naming that
+   machine and supplying the type's own parameters (for `console-test`, one
+   `expect` list per console).
+3. Run it: `./qemu_smoke_riscv64.py console-test run <your-test-name>`.
+
+No code change is needed, a test is pure config.
+
+Adding a new test type
+----------------------
+
+1. Create `mytype/` with `mytype.py` defining a `RiscvQtbTest` subclass: set
+   `type_id`, `description`, and `config_file`, and implement `from_config`,
+   `list_tests`, and `run(vm)`.
+2. Add `mytype/__init__.py` that does `from .mytype import MyType`.
+3. Add `MyType` to `TEST_TYPES` in `__init__.py` so the CLI exposes it.
+
+Unit tests
+----------
+
+The `unit/` directory holds pytest tests of the framework's own logic (config
+parsing, device-tree rendering, QEMU arg assembly). They do not boot QEMU and
+are independent of the CI smoke tests, but they import the framework, so they
+need the prerequisites above plus `pytest`.
+
+Run from the Xen tree root:
+
+```
+python3 -m pytest automation/scripts/qtb/riscv/unit/
+```


--
Baptiste Le Duc | Vates Hypervisor & Kernel Engineer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech