Distributions

A distribution is a build that generates values. In the build system, metadata about distributions can be collected in the values.jsonc files along with attestations:

• An attestation is a cryptographically verifiable statement (ex. "the build produced bundle A at time T") that is signed by a human (ex. you) or a machine (ex. GitHub Actions).

To increase supply chain security guarantees, the build system will reject assets and objects that are produced by humans or machines without attestations that you have explicitly trusted.

The following sources of attestation are recognized:

• A human can sign a build by using an OpenBSD signify key.
• GitHub Actions can sign a build using one of two Supply-chain Levels for Software Artifacts (SLSA) security levels:
  • Level 2: https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/use-artifact-attestations#generating-artifact-attestations-for-your-builds
  • Level 3: https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/increase-security-rating

Unfortunately, GitLab does not yet provide a minimal level of attestation.

The build system maintains a trust store, with the dk OpenBSD signify key for the CommonsBase_Std packages as the only trusted entity by default.

Distributions are sealed

A published distribution is sealed: the exact set of value versions it contains (its bundles, forms and objects, each at a fixed version) is determined when the distribution is built and is signed by the distribution's attestation.

When a consumer imports a distribution, only the value versions that distribution sealed are resolvable. Requesting a module the imported distribution did not publish must fail. Requesting different versions of a module than the distribution sealed must also fail.

Sealing module versions provides the following security feature:

• The signify signature and SLSA provenance attest the precise values the distribution was built with. The attestation guarantees a consumer receives exactly the version that were attested.
• Because a consumer can only build and use the value versions the trusted signer explicitly attested, a compromised mirror, a tampered cache, or a later malicious build cannot introduce an unpublished or substituted module into the consumer's build graph.

A package that already has published releases cannot have a module added to an existing release: the module must be introduced by building and publishing a new distribution version.

Adding to the sealed set

A new distribution version whose MAJOR.MINOR is strictly higher than the latest sealed release may add to the sealed set:

• a brand-new module that no sealed release published, or
• a strictly higher MAJOR.MINOR module version of an already-sealed module.

Already-sealed (module, version) pairs are immutable for every version bump. Patch-only releases (ie. the same MAJOR.MINOR) must not change the sealed set.

Distributed Value Stores

A distribution includes a .zip file of some or all of the value store. Entries in the zipfile must be ./{value_id}; for example, ./vnmfdhn7lw4wepx2qiunrmgm4o5lx4wwsf2yfj7xyxggkg5kdsltq.

The following values must be present:

• the "j" values file for any values.json with a form or bundle having a library identical to the distribution library id
• the "w" parsed values ast for any values.json with a form or bundle having a library identical to the distribution library id
• the "o" object file for any object having a library identical to the distribution library id, except an object whose slot the build system produced by expanding the execution_abi wildcard, when the distribution ran with a target ABI different from its execution ABI

That exception is a host slot, in the sense given under Object Slots. The value was computed while cross targeting, so the slot it landed under is the one the host's own native run publishes its artifacts under. A release assembled from several runs would otherwise bind one key to two different values, and a consumer resolving that key would receive whichever run was assembled last, with nothing reporting that a choice had been made.

A run whose target ABI differs from its execution ABI publishes every object whose slot came from a literal execution_slot, including one whose terms spell the execution ABI. That run is the only producer of such a value, so the key it binds stays bound to one value.

An object that carries no record of how its slot was written is excluded when the final term of its slot equals the execution ABI exactly, which is the test the o value id applies to the target ABI in Value Id Formulas. A value store carries that record for every object a run computes, so a slot's terms decide the question only for a value a distribution reuses from a store that lacks the record.

The exclusion is about publishing only. The value is still computed and its value id is unchanged. The run is a cross build and the object's slot does not already name the target, so the object id already hashes the target ABI (the XT term in Value Id Formulas).

The following values will be ignored if present:

• the "c" constants

A unoptimal implementation can simply zip up the value store directory, but done frequently that leads to wasted bandwidth, storage and lengthier builds.

An optimal implementation only includes the necessary values.

A distribution may be assembled from more than one producing run. A producer must not publish one key id from more than one run: within a distribution a key id binds to exactly one value id.

An importer offered a key id bound to more than one value id must report the divergence, naming the key, its slot, every value id offered and the one it bound. It binds the key to the first candidate the value index walk offers, so the result does not depend on how the walk was scheduled. Reporting is required because the importer cannot tell the producing runs apart: nothing a distribution carries names the run that published a value, so an importer can detect the conflict and cannot resolve it.

OpenBSD signify keys

The security policy is described in Securing OpenBSD From Us To You. Signify builds keys are automatically created specific to the user (you) on a machine. The private keys should not be shared except among a small set of trusted co-signers (ex. your manager and a peer).

Any OpenBSD signify key and any GitHub repository encountered by the build system is automatically denied. However, you are prompted if you want to accept or deny the key or repository, with the default as deny.

A consumer can record a durable acceptance for a package name before any import, so the next import of that package accepts the producer key it presents without a prompt. The acceptance optionally pins the expected public key obtained out-of-band; an import whose producer key differs from the pin is denied, so a first import over a compromised channel is denied rather than accepted on first use. An acceptance is an authenticity decision and never grants a package's rules a capability.

When accepting an OpenBSD signify public key embedded in a values.json build file, the

• accompanying continuations that were signed with the private key are accepted. These continuations are public keys for the next version, and imported once and never overwritten.
• accompanying builds that were signed with the private key are accepted
• all builds signed with the private key that with packages that share the major + minor parts of the accompanying build are accepted

Once a major and minor component of a package has been signed, no lower versioned distribution key can claim:

• that major and minor component
• any later versions to that major and minor component

That may be difficult to understand without an example. We'll use the Key Rotation example from Securing OpenBSD From Us To You, where OpenBSD is about to release OpenBSD version 5.6. They:

• have an existing public/private key pair for version 5.6
• have generated public/private key pairs for version 5.7 before 5.6 is released
• have generated public/private key pairs for version 5.8 before 5.6 is released

In the build system the version 5.6 would be a distribution; let's say OpenBSD_Std@5.6.0:

{
  "distributions": {
    "id": "OpenBSD_Std@5.6.0",
    "producer": {
      "openbsd_signify": {
        // key from https://ftp.eu.openbsd.org/pub/OpenBSD/signify/openbsd-56-base.pub
        "public_key": "untrusted comment: openbsd 5.6 base public key\nRWR0EANmo9nqhpPbPUZDIBcRtrVcRwQxZ8UKGWY8Ui4RHi229KFL84wV"
      }
    },
    "continuations": {
      "attestation": {
        "openbsd_signify": {
          // this is a (fake) signature of SHA256(plaintext), where
          // plaintext is `{"5.7":"...","5.8":"..."}` (see below)
          "signature": "untrusted comment: signature from signify secret key\nRWTAeKJJ1MTF3UpxzBCu6NaM6HPJNTj5CZ+M5XNJKNeEHBLQSsstzHGbSo8rPYNgw3Z98pN7WKiIwBIyRrKuIdKBRA6qlaci6wI="
        }
      },
      "continuations": {
        // https://ftp.eu.openbsd.org/pub/OpenBSD/signify/openbsd-57-base.pub
        "5.7": "untrusted comment: openbsd 5.7 base public key\nRWSvUZXnw9gUb70PdeSNnpSmodCyIPJEGN1wWr+6Time1eP7KiWJ5eAM",
        // https://ftp.eu.openbsd.org/pub/OpenBSD/signify/openbsd-58-base.pub
        "5.8": "untrusted comment: openbsd 5.8 base public key\nRWQNNZXtC/MqP3Eiu+6FBz/qrxiWQwDhd+9Yljzp62UP4KzFmmvzVk60"
      }
    }
  }
}

In the same distribution in values.jsonc we also must have at least one build:

{
  "distributions": {
    "id": "OpenBSD_Std@5.6.0",
    "producer": { /* ... */ },
    "continuations": { /* ... */ },
    "build": {
      "attestation": {
        // this is a (fake) signature of SHA256(plaintext), where
        // plaintext is `{"modules":...,...}` (see below)
        "openbsd_signify": "untrusted comment: signed by key c078a249d4c4c5dd\nRWTAeKJJ1MTF3UpxzBCu6NaM6HPJNTj5CZ+M5XNJKNeEHBLQSsstzHGbSo8rPYNgw3Z98pN7WKiIwBIyRrKuIdKBRA6qlaci6wI="
      },
      "build_to_sign": {
        "modules": ["DkExe_Std.Form@1.0.202501010000"],
        /* other build fields described in the mlfront-values.json schema */
      }
    }
  }
}

The OpenBSD_Std@5.6.0 public key has signed for DkExe_Std.Form@1.0.202501010000, but can also sign for any DkExe_Std.Form@1.0.* (and no others!) since those all share the major and minor number of the DkExe_Std.Form in values.jsonc.

The OpenBSD_Std@5.7.0 public key can sign for either DkExe_Std.Form@1.0.* or a later version (ex. DkExe_Std.Form@1.1.*), but cannot sign for both. That is, once a OpenBSD_Std@5.7.0 public key is seen to accompany DkExe_Std.Form@1.1.*, the association OpenBSD_Std@5.7.0 may-sign DkExe_Std.Form@1.1.* is stored in the trust store, and it can no longer sign any other major + minor version.

Most important, the OpenBSD_Std@5.7.0 public key cannot sign for an earlier version (ex. DkExe_Std.Form@0.7.*) than OpenBSD_Std@5.6.0.

The net effect is a tendency of the trust store to increase versions over time, so that these OpenBSD signify keys have an implicit rotation policy as these versions increase. You choose how many keys you want to keep alive at any time (the current key plus the number of keys in your continuations), and do a rotation by doing a new build using a key from one of your continuations.

So, let's say you keep 2 continuations alive (plus the current key) like OpenBSD does. When you start using a key from one of your continuations (ex. OpenBSD_Std@5.7.0), you should will keep N-1 = 1 keys in rotation (ex. "continuations" should contain OpenBSD_Std@5.8.0 still) and create one new key (ex. "continuations" should have a new OpenBSD_Std@5.9.0 key).

Distribution versioning

The first trusted release that claims a MAJOR.MINOR establishes its key, and that association is imported once and never overwritten.

The implication is that distribution keys must be rotated for a new MAJOR.MINOR distribution package version.

• A release within the current line (a new MAJOR.MINOR.PATCH, ex. the MAJOR.MINOR.<UTC timestamp> tags of a dk package) is signed by the line's existing secret key.
• A release on a line listed in the current line's continuations is signed by that pre-signed key. The consumer accepts it through the continuation chain with no new trust decision. The producer must possess the continuation's secret key to perform the release.
• A release on a line absent from every trusted release's continuations has no chain to it. Every consumer is prompted to accept its key as if the producer were new.

Stating the engine version floor

A release records min_dk_version, the oldest dk engine that can consume its wire format. The producing engine stamps the field from a constant of its own build, so the floor states the format the release's stores are written in.

• The value is a version string in either the MAJOR.MINOR.PATCH.BUILD form or the MAJOR.MINOR.PATCH+rev-N form, compared component by component with missing trailing components read as 0. A consumer version that is an exact prefix of the floor is the development tip of that line and is accepted.
• import and inspect refuse a release whose floor is above the running engine. restore treats such a release as an unusable build cache, reports it, and builds cold.
• A release carrying no min_dk_version field is refused on those same terms. Its producer predates the field, so the running engine has no statement that the release is readable.
• min_dk_version is excluded from the canonical form; it can be changed without changing the value id.
• A partial release stamps a higher floor than a full release from the same engine, and combine stamps the highest floor among the parts it joins.

GitHub SLSA Level 2

When accepting a GitHub repository (SLSA Level 2), the:

• accompanying builds attested by GitHub are accepted
• all builds attested by GitHub are accepted

The build system will download the GitHub CLI (using the default trusted CommonsBase_Std library) to do verification of the builds.

GitHub SLSA Level 3

When accepting a known, vetted GitHub Actions script (SLSA Level 3), the:

• accompanying builds attested by GitHub are accepted
• all builds produced by the GitHub Actions scripts that are attested by GitHub are accepted

The build system will download the GitHub CLI (using the default trusted CommonsBase_Std library) to do verification of the builds.

Distribution Scripts

🚧missing docs: make a section on cram tests independent of distributions

When writing distribution scripts:

1. Running one rule in a script module will bring in the entire script module. In particular, if a distribution script runs a function rule from a *.values.lua scriptmodule, the entire *.values.lua scriptmodule is distributed. Function rules should follow the F_<Name> naming convention to avoid colliding with UI rule names in the same scriptmodule.

   It is best to run every function rule for lightweight testing during distribution.

2. Use ${CONFIG} for files in the cram test directory.

3. Use ${RUNTIME}/<unique path to cram test> for -f and -d options since outputs are relative to the current dir like run-function, dialog, and exec. It is set unique to the cram test. The <unique path to cram test> is to avoid race conditions on Windows where Windows Defender (etc.) may not make a file visible or rewritable immediately after creation.

4. Declare each static rule input in the rule's declareoutput response.