App::FuguVM::DiskCache(3p)

Perl Library Manual

App::FuguVM::DiskCache(3p)

NAME

App::FuguVM::DiskCache - cache of installed OpenBSD disks

SYNOPSIS

    use App::FuguVM::DiskCache;
    my $cache = App::FuguVM::DiskCache->new('~/.cache/fuguvm');
    my $key   = $cache->key($vm_config);
    # Boot from a previous installation
    if (my $hit = $cache->lookup($key)) {
        $disk->create($name, undef, $hit->{base});
        $state->set_root_password($hit->{meta}{root_password});
    }
    # Publish a freshly installed disk
    $cache->store($key, $disk_path, { root_password => $password });
    for my $entry (@{ $cache->list }) {
        printf "%s %d bytes\n", $entry->{key}, $entry->{size};
    }

DESCRIPTION

An installation of OpenBSD under TCG emulation takes tens of minutes. This module keeps the result, thus later runs do not do the installation again. The result is a clean, compacted copy of the disk. The module makes this copy when the installation is complete. Later VMs use this copy as the backing image of a throwaway qcow2 overlay.

The entries are in the configured "cache_dir", adjacent to the proxy cache that holds the miniroot and the install sets:

    <cache_dir>/installed/<key>/base.qcow2              mode 0400
    <cache_dir>/installed/<key>/meta.json               mode 0600
    <cache_dir>/installed/<key>/snapshots/<name>.qcow2  mode 0400
    <cache_dir>/installed/<key>/snapshots/<name>.json   mode 0600

The entries are immutable and write-once. "store" builds a full entry in a sibling ".tmp.*" directory, and then it publishes the entry with a rename of that directory. Thus a reader never sees the base image of one installation adjacent to the metadata of a different installation. Such a mismatch looks live, but it then stops every later boot, because the root password does not open the image.

Cache keys

"key" returns "<version>-<arch>-<hash8>". "hash8" is a truncated SHA-256 over each input that shapes an installed disk, and the record follows the "install_mode" of the configuration, because each mode shapes the disk with different inputs:

expect: The OpenBSD version, the architecture, the disk size, the mode, the contents of the install.exp script that App::FuguVM::Console resolves, and the contents of share/fuguvm/cache-generation.
autoinstall: The same inputs, with two differences: the script is autoinstall.exp, and the contents of the response file join the record. The response file answers every installer question, so it shapes the disk as much as an expect script does. Two response files give two keys, two entry directories and two immutable base images.
import: The OpenBSD version, the architecture, the mode, and the generation file. No script installed an imported image, so the record holds no script digest, and an imported entry survives a change to a shipped expect script. An overlay inherits the virtual size of its base, so the record holds no disk size either, and each host of a fleet derives one key.

The generation file is a counter. Increase the number in this file when the install driver changes in a way that the script hash cannot see. One example is a change in how App::FuguVM::Guest or App::FuguVM::Console drives the installer.

The counter is a data file, not a constant in this module. Thus a continuous-integration cache key can hash the file too. Keep the file content to the bare number: the hash covers the whole content, and an edit to a comment changes the key.

The memory settings and the port settings do not shape the disk, and the key intentionally excludes them. Thus a change to these settings hits the same entry.

Snapshots

A snapshot is an immutable, named layer over the base image of an entry. Its purpose is to cache states that fuguvm itself does not know. One example is a provisioning script that caches its "guest packages installed" state. The mechanism is in this module. The policy stays in the callers: they select the states that get a cache.

The snapshots are inside "installed/<key>/". Thus, when a base becomes invalid - through a new key, or through "cache clear" - the snapshots of that base also become invalid automatically.

"snapshot_store" flattens the working disk onto "base.qcow2". It does not copy the disk. Thus each snapshot is a direct child of the base:

    base.qcow2  <-  <name>.qcow2  <-  disk.qcow2

A copy keeps the backing-file header of the working disk verbatim. That header is correct only while the disk hangs directly off the base. After a restore, the disk hangs off a snapshot. Thus a copy stacks chains without a limit, or it publishes a qcow2 that names itself as its own backing file. The second result occurs when a caller saves the same name again. A normal second run does this.

Because "snapshot_store" flattens the disk, no snapshot is the parent of a different snapshot. Thus "snapshot_remove" cannot make a snapshot an orphan.

The working disk must come from a stopped VM. A live overlay is not consistent.

METHODS

new($cache_dir)

The constructor creates a cache over $cache_dir. It expands a "~" at the start of the path.

installed_dir

entry_dir($key)

base_path($key)

These methods return locations. The paths do not have to exist.

key($vm_config)

The method derives the cache key for a VM configuration hash. It returns "undef" when it cannot read an input, and when the configuration carries no architecture. The caller then has no key, and thus no cache.

Two architectures give two key strings and two entry directories. Thus one cache holds a base image for each architecture, and neither entry can overwrite the other.

lookup($key)

The method returns "{ key, dir, base, meta }" for a complete entry. In other cases, it returns "undef". A half-written entry is a miss, not an error.

store($key, $disk_path, $meta)

The method compacts $disk_path into the cache as the base image for $key, and writes $meta adjacent to it. Put the guest "root_password" in $meta. The method returns the path of the base image. It returns "undef" on a failure, and a try to overwrite a populated key is one such failure. The caller then degrades to a standalone disk.

lock_entry($key, $timeout)

The method returns an open handle with an exclusive flock(2) on the lock file of $key: "<cache_dir>/installed/.lock.<key>". The wait blocks, with a deadline of $timeout seconds (default 3600). The method returns "undef" on the deadline, and "undef" when the file cannot open.

The lock releases when the handle closes, or when the process exits. Thus a stale lock file blocks nothing.

The lock serializes the first population of one entry. The caller takes the lock, looks the key up a second time, and installs only on a second miss. Every project that shares the cache directory shares the lock. The lock does not replace the write-once rule of "store": a run that lost the lock to its deadline still cannot publish a second entry.

The file name starts with a dot. Thus it cannot collide with an entry: "list" reads only a directory whose name has no leading dot, and "sweep_temp" removes only a ".tmp." directory.

list

The method returns each complete entry, newest first, as "{ key, dir, base, meta, size, created_at, snapshots }".

remove($key)

The method deletes an entry and all data under it. It returns true when the entry is gone.

sweep_temp

The method removes the ".tmp.*" trees that an interrupted "store" left. This includes the trees of earlier processes. The method returns the count of removed trees.

key_for_path($path)

The method returns the key of the entry that holds $path. The path points to a base image or a snapshot. The method returns "undef" when $path is outside the cache. The method answers this question: "Which cached image is the base of this working disk?"

snapshot_dir($key)

snapshot_path($key, $name)

These methods return locations. The paths do not have to exist.

valid_snapshot_name($name)

The method returns true when $name is usable. A name becomes a file name inside the cache. Thus a name must start with an alphanumeric character, it must hold only word characters, dots, and dashes, and it must have a bounded length.

snapshot_store($key, $name, $disk_path, $meta)

The method flattens the stopped working disk at $disk_path onto the base image of $key, and publishes the result as the named layer. Put the state fields that the disk holds in $meta: "installed" and the installed SSH public key. With these fields, a restore can reseed App::FuguVM::State. The method copies the root password from the metadata of the base itself, and it does not trust the caller for this value. The method returns the snapshot path, or "undef" on a failure.

When a caller saves the same name again, the method replaces the snapshot. This is the normal second run of a provisioning script.

snapshot_lookup($key, $name)

The method returns "{ key, name, path, base, meta }" for a snapshot whose image, metadata, and backing chain all resolve. In other cases, it returns "undef". A snapshot with a removed base is a miss, not an error. Thus a caller can provision from scratch, and it does not fail hard.

snapshot_list($key)

The method returns the sorted snapshots of an entry, as "{ name, path, size, created_at, meta }".

snapshot_remove($key, $name)

The method deletes a snapshot and its metadata. Deletion in any order is safe, because each snapshot is a direct child of the base, never of a different snapshot.

SECURITY

meta.json is mode 0600, and it holds the generated guest root password, the same secret that the VM state directory keeps. The cache makes the life of this secret longer: the password stays after "fuguvm destroy", and it changes only when the base key changes.

The guest permits root login with a password. QEMU binds the forwarded SSH port and the serial console port of the guest to the configured "bind_address". The default is 127.0.0.1. With "bind_address 0.0.0.0", the password is not a localhost-only secret. See fuguvm(1).