Re: [Xen-devel] Architecture for dom0 integrity measurements.

["Dr. Greg Wettstein" <greg@xxxxxxxxxxxxxxxxx>]

On Jan 12,  5:25pm, Daniel De Graaf wrote:
} Subject: Re: [Xen-devel] Architecture for dom0 integrity measurements.

Hi Daniel, thanks for taking time to respond.  Hope your week is going
well.

> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxx] On Behalf Of Dr. Greg Wettstein
> >> Sent: Saturday, January 10, 2015 10:59 PM
> >> To: xen-devel@xxxxxxxxxxxxx
> >> Subject: [Xen-devel] Architecture for dom0 integrity measurements.
> >>
> >> Hi, I hope the weekend is going well for everyone.
> >>
> >> We have been watching the discussions on the list over the holiday on the
> >> refinement and enhancement of the TPM architecture for Xen, including
> >> support for TPM 2.0.  We are active in measured platform development and I
> >> wanted to pose what is perhaps a philosophical question to everyone.
> >>
> >> Our systems boot from a hardware root of trust via TXT and we heavily 
> >> leverage
> >> the Linux Integrity Measurement Architecture (IMA) for mutual remote
> >> attestation.
> >
> > Is it based on TBoot / OpenAttestation ?
> >
> >> I understand the motivation for running the TPM hardware
> >> manager in an IO emulation domain but unless I miss something in the 
> >> current
> >> discussions, this architecture precludes the ability of the dom0 kernel to
> >> physically access the TPM which in turn prevents
> >> dom0 from implementing a hardware referenced measurement state via IMA.
> >
> > Graaf can answer it.
>
> Ideally, domain 0 would use its own vTPM to record IMA measurements.
> However, it is not currently possible to use IMA with a vTPM that is
> started after the domain 0 kernel has booted.  In addition, the
> current vTPM emulator requires working disk I/O and a working vTPM
> Manager domain before it will accept commands.

For those reading along at home, a quick summary of the current TPM
virtualization architecture, Daniel correct me if I'm wrong.

There can be only one physical projection of the TPM hardware
resources by the hypervisor.  This is done to a management process
(vtpmmgr) which is an application running in a small isolated OS
instance (mini-os).  This process is responsible for issueing the
ordinals (commands) to the physical TPM which are needed to use the
security/trust guarantees provided by that hardware.

Separate isolated OS instances are created for each 'virtual TPM'.
These OS instances run TPM simulation software (Berlios) whose
functionality is presented to standard OS guests (Linux/Windows)
through a hypervisor mediated shared memory interface.

For Linux integrity management to work the kernel needs to have access
from the moment it starts to a TPM in order to add the measurements of
any resources it accesses, which are determined to being security
sensitive by the security policy, to the TPM.

So we are in a 'Catch-22'.

The dom0 kernel is responsible for starting the virtual TPM
infrastructure but needs access to a TPM if its own security status is
to be measured.  Since properly starting the virtual TPM
infrastructure is a security critical operation the virtual TPM's
presented to the guests are technically insecure unless the integrity
of dom0 is measured/monitored.

In a standard Xen implementation dom0 is usually a regular Linux
distribution which is a pretty big piece of software to leave
unmeasured, or to even measure as far as that goes.

The challenge is how to address this 'Catch-22'.

> Architecture 1:
>
> Domain 0 boots with access to the physical TPM (via tpm_tis) and
> uses it to store IMA measurements.  During boot, the vTPM Manager
> domain and a vTPM for domain 0 are started.  Since the vTPM Manager
> currently uses locality 2 while Linux uses locality 0, concurrent
> access is possible without needing a cross-domain lock.  The TPM
> driver in both minios and Linux will wait long enough for a short
> command (such as a PCR extend) to complete when submitting their
> commands.
>
> Once the vTPM has been started, the physical TPM can be
> hot-unplugged from domain 0, causing future TPM commands to be
> directed to the vTPM.  The vTPM for domain 0 should be configured to
> mirror hardware PCRs on startup so that the early IMA measurements
> are reflected in the vTPM.
>
> One problem with this: the process of switching from physical to
> virtual TPM needs to happen during a quiescent period for IMA
> measurements.  This will result in the vTPM's PCR10 containing the
> full IMA log and the pTPM's PCR10 containing an earlier, truncated
> version.  If IMA events happen during the switch, there will be some
> measurements only present in the hardware TPM with the remaining
> only be present in the vTPM.  While it is possible to reconstruct
> the values of each PCR during verification, this is more complex and
> should be avoided if possible.  In either case, a verifier should
> check both physical and virtual TPM's PCR10 values for validity
> against the dom0 IMA log.

Summary.

dom0 gets to control the physical TPM and record measurements with it
until enough infrastructure gets established to start vtpmmgr and a
vTPM which the kernel will use from that point forward.

We need to make sure the physical and virtual TPM's agree with another
as part of the handoff.  The kernel cannot access any resources which
are deemed security sensitive during the handoff.

> Architecture 2:
>
> A more disaggregated system can use a domain 0 that uses no
> permanent state: its root filesystem would be a ramdisk optionally
> supplemented with a dm-verity disk image.  This type of domain does
> not need to use IMA, and so can avoid any TPM access until both the
> vTPM Manager and vTPMs are created.  Normal domain 0 functions would
> actually be performed by a toolstack domain distinct from domain 0.

Summary.

Implement dom0 as a very small and minimal footprint 'security
supervisor', whose integrity is not measured, which starts the vtpmmgr
and vTPM's.  Punt the virtual machine support infrastructure into
separate and isolated domains (guests) which use the vTPM's to measure
themselves.

> This is the architecture that I have primarily used in testing.

We have that architecture implemented on our platform with the
exception that we are not, yet, projecting a virtual TPM
implementation into guests.  Our security supervisor retains control
of the physical TPM and does integrity measurement on itself and what
it manages.

We wrote the HPD driver:

ftp://ftp.enjellic.com/pub/hpd/hpd_driver-1.0beta.tar.gz

Which implements a block device on top of the hugepage infrastructure
to support this model.

But we are now at the point where we need to project virtual TPM's
into guests.  Our security supervisor is really small and uses libxl
and by extension libxc directly but the necessary support libraries
and infrastructure makes us hesitant to run all this without integrity
measurement.

> Architecture 3 (requires changes to the vTPM domain):
>
> A domain builder (stub domain) creates an initial set of domains
> including Xenstore, vTPM Manager, hardware domain, and a vTPM for
> the hardware domain.  The vTPM must be capable of performing PCR
> extend operations and other commands prior to being able to access
> its disk image, which will be attached (xl block-attach) during the
> hardware domain's boot process.
>
> Boot event order here:
>   1. Xen startup (pre-dom0)
>   2. Domain builder starts as dom0
>   3. Creation of xenstore, vtpmmgr, vtpm-hwdom, hwdom
>   4. Wire up vTPM connection for hwdom in Xenstore
>   5. Unpause vtpm-hwdom and hwdom
>   6. Hardware domain boots, uses vtpm-hwdom for IMA logging
>   7. Disk images for vtpmmgr and vtpm-hwdom attached
>   8. Unpause vtpmmgr domain
>   9. Start trousers and/or use kernel trusted-keys operations

Summary.

The hypervisor runs a 'security supervisor' in place of a standard
dom0 with enough functionality to initiate and run a cohort of
disaggregated guest domains.  The vTPM supporting the security
supervisor needs to have sufficient functionality, without requiring
disk operations, to support adding measurements until enough
infrastructure gets running to plug storage into the vtpmmgr and the
security supervisor's/dom0 vTPM.

> >> Others may disagree but I wouldn't even contemplate delivering an
> >> integrity certified platform without including all of the dom0
> >> infrastructure into the platform measurement status.  We heavily
> >> leverage the current 4.4.x vTPM implementation for testing and
> >> development and the documentation states clearly to not integrate
> >> TPM/TIS support into the dom0 OS.
> >
> > Ditto.

> Right, measuring domain 0 is important in getting useful assurance
> from a platform.  Using other TPM functionality within domain 0 may
> also be useful, which is why I think making a vTPM available to
> domain 0 is a good solution.  The recommendation in the documents is
> really about access to the physical TPM, which interferes with the
> vtpmmgr domain.

There seems to be uniform consensus that dom0 needs integrity
measurement.  Our security supervisor uses PCR sealed NV-RAM so we
have a need for functionality beyond extension as well.

> >> The obvious model is to run a software TPM simulator in dom0 and
> >> have the vTPM I/O domains link to that.  We are heavily invested
> >> in IBM's software TPM simulator and have been tossing around the
> >> idea of building up a proof of concept based on that.  I wanted
> >> to make sure we were not misunderstanding anything with the
> >> current or proposed architecture before we invest the resources.
> >
> > IBM's software TPM simulator, is it libtpms?  For all I know, the
> > libtpms is a library that targets the integration of TPM
> > functionality into hypervisors.  In this mode, libtpms is dynamic
> > linking library, so there is no root of trust.  If you really want
> > to enable it, I have some Suggestion.
> >
> > 1. vTPM I/O domains is now needed in this mode, QEMU can implement
> >    another TPM
> >
> > Backend to link libtpms. Try to refer to
> > http://lists.nongnu.org/archive/html/qemu-devel/2013-11/msg00674.html
> >
> > 2. Enabling seabios for HVM virtual machine.
> > Refer to patch ' vTPM: add TPM TCPA and SSDT for HVM virtual machine'
> > And https://github.com/virt2x/seabios2
>
> Using libtpms does not prevent having a root of trust, but it does
> greatly increase the size of the required trusted computing base to
> include all the software with sufficient privilege in domain 0 (and
> relevant configurations).  This can be measured by a properly
> configured IMA policy, although it is more difficult to perform
> local attestation with IMA (the exact ordering of IMA records is
> often not deterministic, meaning you cannot reasonably seal secrets
> to a state that includes the IMA PCR).

The majority of our engineering time and resources has been expended
on the issue of IMA determinism.  We have made enough progress that I
believe it is tractable to consider doing a TPM simulator in the
context of our security supervisor and then extending vTPM instances
into guests.

> One of the features of Xen's vTPM architecture is that it is
> possible to have a domU that trusts as little of domain 0 as
> possible.  With a domain builder to bootstrap the domU into a
> trusted state, this does not need to include any management software
> or even the hardware domain's kernel.  However, this level of
> independence is not useful to most guests because it requires that a
> guest perform its own disk/network encryption, which conflicts with
> useful features such as cloning a guest.  Migration also becomes
> more complex (requiring a separate migration domain).

Our Golden Retriever Izzy and I have spent the last two days walking,
dodging coyotes that have run out of jack rabbits to eat and thinking
intensely about all this.

I tend to find the notion of increasing disaggregation and isolation
of functionality into domains increasingly attractive.  However there
is the challenge of managing the economic balance between complexity,
engineering costs and the amount of security people need and more
importantly are willing to pay for.

In contrast to where a lot of TXT and DRTM based technology is focused
we work in the 'imbedded' space, whatever that means now when one has
access to quad-core processors and gigabytes of RAM..:-) We ended up
implementing virtualization in our supervisor not as much for security
isolation but because it lets us run Windows, another necessary
reality of economics.

So based on your reflections I think we are going to take a two
pronged approach.  Running a multi-headed TPM simulator in the context
of our security supervisor seems to be a no-brainer, at least thats
what Izzy concluded.

Long term I think your approach #3 is consistent with our platform and
where all this needs to go.  But there is a lot of work that needs
to be done.

> - Daniel
> Daniel De Graaf
> National Security Agency

Thanks for jumping in with your reflections and all of your effort in
bringing this technology to Xen.

Have a good weekend.

Greg

}-- End of excerpt from Daniel De Graaf

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@xxxxxxxxxxxx
------------------------------------------------------------------------------
"All parts should go together without forcing.  You must remember that
 the parts you are reassembling were disassembled by you.  Therefore,
 if you can't get them together again, there must be a reason.  By all
 means, do not use a hammer."
                                -- IBM maintenance manual, 1925

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel