CommonsBase_Remote 0.2.20260806033025

dk remote-execution rules that let dk0 remote ... run a dk command on a GitHub Actions runner and return its result.

CommonsBase_Remote provides the GitHub orchestration rule (CommonsBase_Remote.GitHub) consumed by dk0 remote .... It snapshots the caller's workspace into an isolated commit directory, encrypts any local assets for a signify-signed per-repository age session key, and pushes tagged prereleases that drive a generated GitHub Actions workflow; it then waits for the run, downloads the combined result, and propagates the remote exit code.

GitHub remote dry-run rule

The initial package exports the GitHub remote orchestration rule in dry-run
mode. Running the free rule here distributes the entire scriptmodule.

  $ run-function CommonsBase_Remote.GitHub.F_DryRunPlan@0.2.0 -f ${RUNTIME}/GitHub-0.2.0.plan.txt dry_run=true timestamp=20260527165106 repo=github.com/OWNER/REPO cmd=get-bundle commandvsl='get-bundle MlFront_Std.Sample.Abc@1.0.0' 'argv[]=get-bundle' 'argv[]=MlFront_Std.Sample.Abc@1.0.0'
  \test(pass)
  \dk.asset(path: "dry-run.txt", value-id: "aw63b2zkamczbsd5f4ndz23hlxh5aiygucc5xu3eo4v5nlcyml4mq", byteSize: "890")\;

The remote command (host side)

`dk0 remote UI_MODULE@VERSION [OPTION=VALUE...] [REPOSITORY] INNER_COMMAND...`
is thin sugar in the `dk0` host (`ShellRemote.ml`) over a plain
`dk0 run <rule>` of this orchestration rule. The host does not run the inner
command locally; it only forwards it. Concretely, `remote`:

1. **Resolves the rule.** The `UI_MODULE@VERSION` is expanded to the full rule
   id: a bare name like `GitHub@0.1.0` becomes
   `CommonsBase_Remote.GitHub.Run@0.1.0` (the `_Remote` library suffix and the
   `.Run` rule are added when missing).
2. **Splits arguments.** Leading `OPTION=VALUE` pairs are separated from the
   inner command, an optional `REPOSITORY` before a recognized inner command is
   detected, and the inner command name is checked against the supported list
   (`get-object`, `run-object`, `run-function`, `run-rule`, `enter-object`,
   `merge-object`, `get-asset`, `run-asset`, `get-bundle`, `test`, `lua`,
   `dialog`, `exec`, `run`).
3. **Computes a requested trace key** for commands that produce
   dependency-bearing values (`get-object`, `run-object`, `run-function`,
   `run-rule`, `enter-object`, `merge-object`). Bundles and assets have no
   dependencies, so they carry no requested trace key.
4. **Builds the rule request parameters** and runs
   `dk0 run CommonsBase_Remote.GitHub.Run@... <params>`, where the params are:
   - `cmd=<inner command name>` and `commandvsl=<inner command as one quoted
     line>` (the human-readable audit line),
   - `requested_trace_key_id=...` / `requested_trace_key_description=...` when
     applicable,
   - the forwarded `OPTION=VALUE` remote options, `repo=<REPOSITORY>`, and
     `project_root=<absolute project directory>`,
   - **`argv[]=<token>` for every token of the inner command.** The argv is
     forwarded verbatim; the runner's `dk0` re-parses it and resolves any
     `$(...)`/`${...}` subshells and variables against the **remote** workspace,
     never the local one. Local argument expansion is intentionally not applied
     to the `remote` path.

So everything below happens inside the `Run` rule, driven by those request
parameters.

Execution and target ABI selection

The `execution_abi=` and `target_abi=` remote options choose where the session
runs and what it builds:

- `execution_abi=` picks the GitHub-hosted execution platform. The supported
  values are `Windows_x86_64` (the default; `runs-on: windows-latest`),
  `Darwin_arm64` (`macos-latest`) and `Linux_x86_64` (`ubuntu-latest` inside
  the manylinux container described below).
- `target_abi=` names the ABI of the generated executables and libraries and
  defaults to the execution ABI. When only `target_abi=` is given, the
  execution ABI is derived from it: Windows targets (`Windows_x86_64`,
  `Windows_x86`, `Windows_arm64`) build on `Windows_x86_64`; Darwin targets
  (`Darwin_arm64`, `Darwin_x86_64`) build on `Darwin_arm64`; Linux targets
  (`Linux_x86_64`, `Linux_x86`, `Linux_x86_64_musl`) build on `Linux_x86_64`.
  When the target differs from the execution ABI the rule prepends
  `--target-abi <TARGET>` to the audited argv, so the runner's `dk0` builds
  for the requested target.
- The runner labels and container images deliberately match the dk package
  distribute workflows (`dk0 prepare-version --ci github`). Linux sessions run
  inside a `quay.io/pypa/manylinux_2_28_x86_64` container (glibc 2.28): the
  newest active dated tag is resolved from the quay.io tags API at submit time
  and baked into the generated session workflow (dated tags are pruned from
  quay.io after some months, so a static pin would rot), the maintained
  `latest` tag is the fallback when that API is unreachable, and
  `linux_image=IMAGE` overrides the image entirely. `Linux_x86_64_musl` builds
  on the same glibc container; the target slot's musl cross toolchain comes
  from the package's own values.
- Windows session steps run under native PowerShell (`pwsh`, the
  windows-latest default shell); Linux and macOS steps run under bash. The
  `dk0.cmd`/`dk1.cmd` launchers are never invoked through a Unix shim.

GitHub remote execution flow

The GitHub remote rule is designed so the user's project tree is only an input
source. All remote-session git operations happen in the isolated commit
directory `.dk/r/c` with commands such as `git -C .dk/r/c fetch`,
`git -C .dk/r/c add`, and `git -C .dk/r/c push`. The user's project may have a
different git remote, or may not be a git checkout at all.

Consumer validation workflow

When validating GitHub remote behavior from another project, do not point `dk0`
at an unreleased local checkout with an ad hoc include path like
`-I some/source/CommonsBase_Remote/etc/dk/v`. The reviewed workflow is:

1. (Diskuv owns this step) Release `CommonsBase_Remote` from its package repository with the normal
   commit, tag, push, and GitHub release flow.
2. In the consumer project, import that released package with:

   ```text
   ./dk0 add github-l2 dkpkg/CommonsBase_Remote

3. Run the consumer's dk0 remote ... smoke against the imported release.

That keeps the validation path aligned with how end users consume the GitHub remote package.

The detailed orchestration has three phases:

1. Bootstrap and local staging

   • Ensure the repository exists and has the remote-session branches dk-session-root and dk-session-<i>.
   • Ensure the repository has trusted recipient metadata:
     • DK_SESSION_KEY stays in a GitHub secret.
     • dk_session_recipient and dk_session_recipient_sig stay in GitHub repository variables.
     • The local orchestrator verifies the recipient signature before encrypting any staged payload.
   • Populate the isolated commit directory .dk/r/c from the user project, including copies of:
     • dk.u
     • dk0 and dk0.cmd
     • optional etc/dk/{d,i,v}
   • Use .dk/r/c/t/{k,c,d} as the isolated key, cache, and data directories for local dk0 operations in the commit directory.
   • Commit the signify verify public key into .dk/r/c so the GitHub runner can verify signed manifests without depending on GitHub variables for that key.
   • Run local dk0 update inside .dk/r/c to materialize the workspace inputs needed by the remote command.
   • If a retry left .dk/r/c dirty, stash that leftover residue before the branch sync step so git pull --rebase can proceed on a clean tree.
   • Rebuild .dk/r/c/.git at the start of each submit attempt: if stale git metadata exists, delete it, then run git -C .dk/r/c init and continue from that freshly initialized repository. This avoids falling back into the outer consumer repository when UI-runtime filesystem probes lag behind the just-created .git directory. Only .dk/r/c may have its git remote or session branches changed by this rule.
   • Resolve the local dk0 wrapper by absolute path from the staged snapshot when invoking it from UI submit code, since the rule runtime cwd may differ from the user project root.
   • When the local submit-side bootstrap needs CommonsBase_Std.Coreutils, invoke the staged snapshot dk0 wrapper while pointing -I at the live project etc/dk/i directory so imported package metadata stays visible before .dk/r/c/etc/dk/i has been fully materialized.
   • Resolve host tools (git, gh) by probing the requested program first, then on Windows falling back to generated wrapper commands that extend PATH with known install directories when direct lookup fails.
   • Identify local assets whose mirrors are all strictly inside the user project directory (via request.ui.assets, which rejects any local origin that escapes the project root) and stage their source files for encryption.

2. Stage release publication

   • Pick a free session branch by shuffling session numbers and skipping any session whose GitHub workflow is already running.
   • Append the audited command line to the session audit file in .dk/r/c.
   • Commit the stage control files AND every staged input in .dk/r/c: the audit line, the argument transport, the generated .github/workflows/dk-session.<i>.yml, the workspace file, the dk0/dk0.cmd and (when present) dk1/dk1.cmd launchers, the etc/dk/{d,i,v} metadata and the ignore/attributes policy. The signed stage index covers all of them, so the runner may execute the verified launchers in the stage phase as well as the exec phase.
   • Stage the CommonsBase_Std / CommonsBase_Build import metadata into .dk/r/c/etc/dk/i (from the isolated host import directory when the consumer has not imported those packages itself) so the runner's get-object calls for the packaged tools resolve.
   • Tag and push 0.1.<TS>-stage.
   • GitHub Actions creates a prerelease for the stage tag whose description is the audited command line.
   • After the stage prerelease exists, the local orchestrator encrypts each local asset's source file for the verified dk_session_recipient and uploads the ciphertext to that release. A committed manifest (covered by the signed INDEX) maps each encrypted blob to its local source path, and the runner is told to --trust-local-package each owning package.

3. Exec publication and result retrieval

   • Commit the execution payload in .dk/r/c, including the copied workspace files, the signed INDEX, the committed signify verify public key, and the workflow file.
   • Tag and push 0.1.<TS>-exec.
   • The runner installs nothing from ad hoc URLs beyond the sha256-pinned mlfront-signify bootstrap. After the signed stage index is verified, the committed dk1 launcher (or dk0 when the consumer project has no dk1) fetches the packaged GitHub CLI (CommonsBase_Build.GitHubCLI@2.92.0) and, when local assets must be decrypted, the packaged age (CommonsBase_Build.Age@1.3.1) with get-object -s Release.<execution abi> into the gitignored .local/ directory — the same reserved namespace the local orchestrator uses for its own tool bootstrap. An actions/cache step keyed on the execution ABI and the pinned tool versions caches .local/gh, .local/age and the launcher data home .local/dkdata across session runs.
   • GitHub Actions verifies INDEX.sig and the INDEX checksums with the committed signify public key, then removes that public key (the matching secret is never shipped) so the runner's dk0 generates its own ephemeral build keypair, runs the requested value command or test/lua/run command, and writes a single combined result file dk-session-result.txt (the remote exit code plus delimited stdout/stderr — one always-non-empty file, because GitHub cannot upload zero-byte release assets). It uploads that file to the exec prerelease.
   • The local orchestrator waits for the exec workflow run matching the pushed commit (headSha), downloads dk-session-result.txt, prints the remote stdout/stderr, and propagates the remote exit code.
   • For local-asset commands, GitHub Actions decrypts each staged source with the DK_SESSION_KEY secret into its project-relative path (per the signed manifest, rejecting absolute or parent-relative paths) before running, so remote get-asset/run-object/get-bundle of local assets resolve their content. Commands needing only the committed workspace (e.g. lua, test) ship no encrypted assets.

Files needed in the project tree

Files placed into the project tree

The current reviewed layout places only the remote-control directory in the user project tree.

The user's original dk.u, dk0, dk0.cmd, and optional etc/dk/{d,i,v} remain the source inputs. The copies under .dk/r/c are the remote-session payload.