Unified scripts

• Context
• Introduction
• Quick Exploration
• Syntax
  • Unified Script Syntax
  • Metadata Syntax
• Terminology
• Mark-in Kinds
  • Builtin Evaluators
  • Cram Tests
  • Mercurial Unified Tests
  • OCaml Module Script
    • OCaml Module Script Setup
  • OCaml REPL Cram Test
    • OCaml REPL Cram Test Setup
    • Automatically Discovered Dune Options
    • Manually Configured Options
  • dk Build Scripts
• Main Document Kinds
  • Plain Text
  • Markdown
  • OCaml Modules
  • Typst
  • AsciiDoc
• Unified Ecosystem
  • Runners
  • Renderers
  • OCaml Users
  • Supporting a new mark-in kind

Context

dk0's own configuration and publishing files are unified scripts:

• dk.u — the workspace script at the root of a project.
• dist-*/run.u — distribution scripts (these may be located elsewhere)

You work with these unified scripts when you:

• publish your values for others to use — you write the distribution scripts (dist-*/run.u) that package and demonstrate them;
• define workspace assets — you hand-edit dk.u so that your distribution scripts know which source and test files in your project are available to be built
• read and understand a project — dk.u and the distribution scripts show what a project imports, builds, and ships.

The rest of this guide explains the unified script format these files use.

Introduction

Unified scripts combine input and output within the same readable document.

Commands are marked into a document with two spaces and a prompt. For example, we can have the command echo Hello World marked into a .md document with <SPACE> <SPACE> $ <SPACE>:

# My Document
We'll run the "echo" command.

  $ echo Hello World

The power of unified scripts is that they are both readable documents and runnable scripts. Running the script above creates an update that includes the response from the echo command:

# My Document
We'll run the "echo" command.

  $ echo Hello World
  Hello World

Tip

Put the updates into source control (ex. git) and you have a persistent record of how the behavior of the script's commands changed over time.

Every line in a unified script is either:

• the main document (Markdown in the example above), or
• a mark-in command, or
• a mark-in response to a command

Tip

The mark-in commands are not limited to shell commands! This documentation, for example, runs each command through an OCaml REPL interpreter.

After a unified script is run, the mark-in command and responses can be rendered to produce a prettier main document. For Markdown documents, the commands and responses are prettier if they are wrapped in code blocks:

# My Markdown Document
We'll run the "echo" command.

```sh
  $ echo Hello World
  Hello World
```

Because unified scripts always have a main document (ex. Markdown above) and a mark-in command evaluator (ex. a shell command above), you should first read the appropriate main document guide:

Plain Text	AsciiDoc
Markdown	OCaml Modules
Typst

and then an appropriate mark-in guide:

Cram test .t	Mercurial test .t
OCaml REPL cram test .ml.u	OCaml Module Script .ml
dk Build Script .dk.u

Quick Exploration

A .ml.u unified script runs OCaml toplevel commands (the same ones an OCaml developer would run in /usr/bin/ocaml or opam exec -- ocaml). Here are some simple OCaml expressions that have been rendered into Markdown with syntax highlighting:

let _ = 2 + 4 ;;

- : int = 6

let _ =
   let x = "hello" in
   let y = "world" in
   x ^ y

- : string = "helloworld"

If you looked directly in the .ml.u script ... the file you would be editing ... you would see:

   # let _ = 2 + 4 ;;
   - : int = 6

   >>> let _ =
   ...    let x = "hello" in
   ...    let y = "world" in
   ...    x ^ y
   - : string = "helloworld"

A .dk.u unified script runs Lua and shell build commands. After running it, important information about the build is placed back into the .dk.u file:

## CommonsBase_Build.Apparatus@0.1.0

Bundle a small CMake project in the hello-src/ directory
using the Lua command "unified.asset"
  % unified.asset { name="HelloWorld", dir="hello-src" }
  'asset'
  '790'
  'sha256:847c39531962e987ba69983babf15f244735f3d566d4fdcaa9a94c038484415d'

Run the "run-function" shell command to build HelloWorld
with CMake
  $ run-function CommonsBase_Build.CMake0.F_Build@3.25.3 -d t/o/somewhere/
  > assetmodver=CommonsBase_Build.Apparatus.HelloWorld@0.1.0 assetpath=hw
  > gargs[]=-DCMAKE_BUILD_TYPE=Release
  > bargs[]=--config bargs[]=Release
  > iargs[]=--config iargs[]=Release
  > outexe[]=bin/hello
  \test(pass)
  \dk.object(abi: "Release.Windows_x86", value-id: "oey3zcwqi3bodhkdahazywr4ikitvpthpwsveinsixe3lq7zhcfxq")\;

Fetch a published asset with the "get-asset" shell command
  $ get-asset CommonsBase_Build.Apparatus.HelloWorld@0.1.0 -p hw -f ${RUNTIME}/hw
  \test(pass)
  \dk.asset(path: "hw", value-id: "aey3zcwqi3bodhkdahazywr4ikitvpthpwsveinsixe3lq7zhcfxq", byteSize: "790")\;

Notice the odd-looking \test(pass)\dk.object(...)\; and \test(pass)\dk.asset(...)\; text in the output. These are response metadata that renderers can use to make pretty output. A \dk.object records a built object's abi and value-id; a \dk.asset records a fetched asset's path, value-id, and byteSize.

Syntax

Unified Script Syntax

A unified script is a UTF-8 document with markin. A UTF-8 BOM is allowed.

The markin regions are:

• Lines beginning with two spaces, a prompt (ex. $ for cram tests), and a space are commands to be run.
• Lines beginning with two spaces, a continuation (ex. > for cram tests), and a space introduce the second, third and beyond lines in a multi-line command.
  • Alternatively, multi-line commands can end with a terminator (ex. ;; for OCaml)
  • Alternatively, multi-line commands can end if the language has rules to know what a statement is complete (ex. Lua chunks)
• All other lines beginning with two spaces are considered the command response.
• Response lines ending with a space and the keyword (re) are matched as regular expressions if a regular expression matcher is provided in the configuration (not configured today!).
• Response lines ending with a space and the keyword (glob) are matched with a glob-like syntax if a glob matcher is provided in the configuration (not configured today!). The only special characters supported are * and ?. Both characters can be escaped using \, and the backslash can be escaped itself.
• Response lines ending with either of the above keywords are always first matched literally with the actual command response.
• Response lines ending with a space and the keyword (no-eol) will match an actual response that doesn't end in a newline.
• Actual response lines containing unprintable characters are escaped and suffixed with a space and the keyword (esc). Lines matching unprintable responses must also contain the keyword.

Any other line is part of the main document.

Metadata Syntax

Metadata can be placed before the true response (the body) of a command response.

The model for metadata is similar to TeX and Typst. Metadata has a named node with optional attributes and optional arguments:

\name(attr1, attr2:"text", attr3:(d=1))[ arg1 ] [ arg2 \more ]

Node names and attribute names must consist of only:

• lowercase letters a-z
• uppercase letters A-Z
• digits 0-9
• some punctuation marks .!?;'" for node names
• some punctuation marks -_!?;'" for attribute names

The node attributes attr1 (etc.) are separated by whitespace and/or optional commas. Node attribute values, if given, are nested attributes surrounded by parentheses or double-quoted text. Only the \\, \r, \n and \" escape codes are allowed inside the double-quoted attribute value text.

Node arguments are text and/or nested nodes. Only the \\ and \] escape codes are allowed inside argument text.

The EBNF grammar for a command response is:

output          = metadata? body ;

metadata        = topnode* last_markup ;
topnode         = whitespace | top_markup ;
top_markup      = "\\" , markup ;
whitespace      = ( " " | "\t" | "\n" | "\r" )+ ;

last_markup     = top_markup | ( "\\" , ";" ) ;
(* "\;" is a special terminator: it immediately ends metadata scanning;
   the body begins right after "\;" *)

body            = (* everything remaining after the last metadata markup *) ;

markup          = markup_name , attributes? , arglist ;
markup_name     = markup_namechar+ ;
markup_namechar = letter | digit | "."       | "!" | "?" | "'" ;
attr_name       = attr_namechar+ ;
attr_namechar   = letter | digit | "-" | "_" | "!" | "?" | "'" ;
letter          = "a".."z" | "A".."Z" ;
digit           = "0".."9" ;

attributes      = "(" , attribute_list , ")" ;
attribute_list  = ( ws? , attribute , ws? , ","? )* , ws? ;
attribute       = attr_name , ws? , ( ":" , ws? , attr_val )? ;
attr_val        = "(" , attribute_list , ")"    (* nested attrs *)
                | '"' , attrtext , '"'          (* quoted string *)
                ;
ws              = ( " " | "\t" | "\n" | "\r" )+ ;
attrtext        = ( attr_char | attrtext_escape )* ;
attr_char       = (* any character except '"' and '\\' *) ;
attrtext_escape = "\\\\"                        (* literal backslash *)
                | "\\\""                        (* literal double-quote *)
                | "\\n"                         (* newline *)
                | "\\r"                         (* carriage return *)
                ;

arglist         = ( "[" , content , "]" )* ;

content         = node* ;
node            = markup_ref | nodetext ;
markup_ref      = "\\" , markup_name , attributes? , arglist ;
nodetext        = nodetext_atom+ ;
nodetext_atom   = node_char                     (* any non-special character *)
                | "\\" , ( "\\" | "]" )         (* recognized escape → literal char *)
                ;
node_char       = (* any character except '\\' and ']' *) ;

Conventionally the attribute names are kebab case (ex. some-word) like Typst, and node names are lowercase.

The following metadata is recognized by the renderers today:

Metadata	What
\ocaml	Render as an OCaml code block
\ocaml(type: "#show")	Render as an OCaml code block, translating ...
	with (* … *) to make valid OCaml
\markdown	Render as raw Markdown

Terminology

to mark-up: An action to mark arbitrary plain text with information. Confer gingerBill

to mark-in: An action to mark regions inside a document with information. A valid document plus the regions added to a valid document (the "markin") remains a valid document. For unified scripts, the "markin" is a command and the evaluator's response to the command.

evaluator: Code or an executable that, given a command, creates a response. Common evaluators include POSIX/Windows shells and programming language interpreters (aka. REPLs). A service's request/reply REST API can also be an evaluator.

Mark-in Kinds

The standard prompts and continuation/terminators (see syntax) are:

Kind	Prompt	Multiline	Runs
built-in evaluator *	%%%	...	Builtin commands
Cram test .t	$	>	POSIX shell command
Mercurial test .t	$	>	POSIX shell command
Mercurial test .t	>>>	...	Python
OCaml REPL cram test .ml.u	#	;;	OCaml expression
OCaml REPL cram test .ml.u	>>>	...	OCaml expression
OCaml Module Script .ml	module expression	end of expression	OCaml expression
dk Build Script .dk.u	$	>	Build shell command
dk Build Script .dk.u	%	end of Lua chunk	Lua command

Builtin Evaluators

File	Prompt	Multiline	Runs
*	%%%	...	Builtin commands

These builtin commands are primarily used for debugging.

Builtin evaluators are not always available; consult your mark-in kind documentation.

The verbatim command lets you see exactly what the unified script sees.

%%% verbatim
... Repeat after me, please.
... This is verbatim output.
Repeat after me, please.
This is verbatim output.

The metadata command strips out all but the metadata, with possible formatting changes:

%%% metadata
... \barename
... \hasattributes0()
... \hasattributes2(one two)
... \hasarguments1nested[ \hasarguments1plain[ inner text ] ]
... \meta[The last metadata before main content]  This is the main content.
... It should not be shown.
\barename\hasattributes0\hasattributes2(one, two)
\hasarguments1nested[ \hasarguments1plain[ inner text ] ]
\meta[The last metadata before main content]

The metadata' command is similar to metadata, except it produces a more verbose format:

%%% metadata'
... \barename
... \hasattributes0()
... \hasattributes2(one two)
... \hasarguments1nested[ \hasarguments1plain[ inner text ] ]
... \meta[The last metadata before main content]  This is the main content.
... It should not be shown.
(markup barename)ws("\n")(markup hasattributes0)ws("\n")
(markup hasattributes2((attr one), (attr two)))ws("\n")
(markup hasarguments1nested[txt(" ")
  (markup hasarguments1plain[txt(" inner text ")])txt(" ")])ws("\n")
(markup meta[txt("The last metadata before main content")])

Cram Tests

File	Prompt	Multiline	Runs
*.t	$	>	POSIX shell command

The "cram test" is the most widely supported unified script format across the unified ecosystem, but it has the least features and needs a POSIX (ie. non-Windows) shell.

Note

You can use Dune Cram Tests or the Python cram package to run cram tests. Eventually a first-class cram runner may be added to the unified ecosystem for use outside OCaml and Python, but for now jonahbeckford@ dislikes that Windows needs a heavyweight POSIX shell to run cram tests.

Mercurial invented this format for their test scripts. Bitheap (Python) and Dune (OCaml) adopted the format and called their scripts {b cram tests}.

Run any shell command:

  $ echo Twas brillig, and the slithy toves
  > did gyre and gimble in the wabe;
  > All mimsy were the borogoves,
  > And the mome raths outgrabe.
  Twas brillig, and the slithy toves
  did gyre and gimble in the wabe;
  All mimsy were the borogoves,
  And the mome raths outgrabe.

Errors will be reported:

  $ hg this-does-not-exist
  hg: 'this-does-not-exist' is not a hg command. See 'hg --help'.

Mercurial Unified Tests

File	Prompt	Multiline	Runs
*.t	$	>	POSIX shell command
	>>>	...	Python

Note

This kind of mark-in is dormant but documented here for devs extending the unified ecosystem.

After Mercurial created the cram test format, they later added support for inline Python code and multiline commands:

```python
Some Python code can be run as part of the test:

   >>> from datetime import date
   >>> print(date(2020, 1, 1))
   2020-01-01

Split the command across multiple lines if needed:
   >>> for i in range(3):
   ...     print(i)
   0
   1
   2
```

Bitheap (Python cram tests) and Dune (OCaml cram tests) did not adopt the >>> prompt.

OCaml Module Script

File	Prompt	Multiline	Runs
*.ml	module expression	end of expression	OCaml expression

Ordinary OCaml module files can be used without any special markup. However, the toplevel module expressions in the OCaml module file must be formatted. In particular:

• the toplevel module expressions must start at column 1, and
• the first word (ex. ["let"], ["module"], ["open"], etc.) of each toplevel module expression must be followed by at least one ASCII space (0x20).

ocamlformat conforms to the formatting requirements.

When run as a unified script, each toplevel module expression is run through the OCaml toplevel interpreter. Here is an ocamlformat-ed OCaml module, before it has been run:

let lyrics =
  "Everybody step to the left."

let (_ : string) =
  Printf.sprintf "Now let's sing: %s" lyrics

module UnifiedShowExample =
  struct
  let contents () = "This is the contents of the module!"
end

After the script is run (see OCaml Modules), OCaml comments are added to a) top-level let expressions and b) modules whose names are prefixed with UnifiedShow:

let lyrics =
  "Everybody step to the left."
  (* val lyrics : string = "Everybody step to the left." *)[@ocamlformat "disable"]

let (_ : string) =
  Printf.sprintf "Now let's sing: %s" lyrics
  (* - : string = "Now let's sing: Everybody step to the left." *)[@ocamlformat "disable"]

module UnifiedShowExample =
  struct
  let contents () = "This is the contents of the module!"
end
  (* - : string = "This is the contents of the module!" *)[@ocamlformat "disable"]

A UnifiedShow* module is required to have a val contents : unit -> 'a function.

All other module expressions are evaluated but do not generate any comments.

OCaml Module Script Setup

1. Run the UMlModuleRunner executable from the mark-in runners table. You will not need a renderer.
   • Pick UMlModuleRunner command line options from either the Automatically Discovered Dune Options (only if Dune is your build tool) or the Manually Configured Options.
2. Optional: Run a renderer that corresponds to your main document

Note

Your .ml script will not have access to the code in the main package (UMlModuleRunner --package MAINPACKAGE). Instead it will have access to the dependencies of MAINPACKAGE so there are no dependency cycles.

Important

Formatters like ocamlformat may dedent the generated OCaml comments, breaking the unified script syntax. You can avoid breaking the syntax using the UMlModuleRunner --disable-ocamlformat option, which adds the [@ocamlformat "disable"] extension to each command response.

OCaml REPL Cram Test

File	Prompt	Multiline	Runs
*.ml.u	#	;;	OCaml expression
	>>>	...	OCaml expression

A .ml.u script uses the OCaml REPL internally: the same REPL you run with /usr/bin/ocaml or opam exec -- ocaml.

"Toplevel phrases" are the commands you give to the REPL. They are either:

• toplevel directives like #show SomeModule, or
• OCaml module expressions like let x = 1 or module X = struct end, or
• OCaml expressions like 1 + 1

.ml.u scripts accept the toplevel phrases in two different forms:

• the # form mimics the OCaml REPL. Each command ends when a line ends with ;;.
• the >>> form mimics the Python REPL. Each command ends when there is no subsequent continuation line ...

For example, these two forms inside a .ml.u file are equivalent:

# 1 +
  2 + 3 ;;
- : int = 6

>>> 1 +
... 2 + 3
- : int = 6

Important

To keep .ml.u files readable, toplevel directives must use the >>> form.

.ml.u scripts also allow built-in commands (%%%) from the built-in evaluator.

Printing

Toplevel values are printed using OCaml's toplevel printing, except unit values are not printed. You can influence the printing of toplevel values with the >>> #install_printer printer-name directive; confer with the OCaml manual's #install_printer documentation.

>>> ()

>>> 'A'
- : char = 'A'

>>> let hexpp ppf c = Format.fprintf ppf "0x%02x" (Char.code c)
val hexpp : Format.formatter -> char -> unit = <fun>

>>> #install_printer hexpp

>>> 'A'
- : char = 0x41

>>> #remove_printer hexpp

Additionally, anything your code prints using Format.std_formatter will be captured and included in the output.

>>> Format.printf "This will be captured!@."
This will be captured!

However, direct use of stdout or stderr goes to the real standard output and standard error. In particular, if your code uses print_endline or Printf.printf then that output will appear on your console while running the cram test rather than inside the cram test response.

>>> print_endline "This will not be captured in the output."

Renderers

Let's go back to the introductory OCaml example:

# 1 + 2 + 3 ;;
- : int = 6

>>> 1 +
... 2 + 3
- : int = 6

With the Markdown renderer, the two forms will render with valid, syntax highlighted OCaml code blocks. However, the rendering is slightly different:

1 + 2 + 3 ;;

- : int = 6

1 +
2 + 3

- : int = 6

The renderers respect the \ocaml metadata. So directly printing OCaml metadata:

>>> Format.printf "%s@." {|
...   \ocaml\;let x = 1
... |}

should render into an OCaml code block:

Format.printf "%s@." {|
  \ocaml\;let x = 1
|}

let x = 1

and directly printing Markdown metadata:

>>> Format.printf "%s@." {|
... \markdown\;| Key | Value |
... | ---      | ---       |
... | Username | `someone` |
... | Domain   | `aol.com` |
... |}

should render into Markdown:

Format.printf "%s@." {|
\markdown\;| Key | Value |
| ---      | ---       |
| Username | `someone` |
| Domain   | `aol.com` |
|}

Key	Value
Username	someone
Domain	aol.com

Tip

Use #install_printer someprinter, where let someprinter ppf (value : sometype) = Format.fprintf ppf "%s%a" {|\whatever\metadata\you\want\;|} your_real_printer value, to always output metadata that produces a nice render for the sometype type.

The metadata syntax is how toplevel directives like #show render with syntax highlighting. The command:

>>> #show Unit

will be updated to include the #show response:

>>> #show Unit
\ocaml(type: "#show")\;module Unit :
  sig
    type t = unit = ()
    val equal : t -> t -> bool
    val compare : t -> t -> int
    val to_string : t -> string
  end

and render as:

#show Unit

module Unit :
  sig
    type t = unit = ()
    val equal : t -> t -> bool
    val compare : t -> t -> int
    val to_string : t -> string
  end

Toplevel directives

The supported toplevel directives are:

#show_class class-path	#show_class_type class-path	#show_exception ident
#show_module module-path	#show_module_type modtype-path	#show_type typeconstr
#show_val value-path	#show ident	#install_printer printer-name
#print_depth n	#print_length n	#remove_printer printer-name
#warnings "warning-list"	#warn_error "warning-list"

Some toplevel directives need to be given directly to the unified script runner UCramRunner (see OCaml REPL Cram Test Setup):

• #directory "dir-name" and #load "file-name". There are UCramRunner --load <file-name> and UCramRunner --load-with-dune <file-name> options that can be used instead. Examples are available in OCaml REPL Cram Test Setup.
• Transitivity: When --load points at a findlib package archive that is listed in a sibling META file, UCramRunner and UMlModuleRunner will auto-load the same package prerequisites from that META first. If the requested archive is not listed in that META, the runners ignore that --load. Package dependencies named in requires are then loaded transitively from their own META files, and already loaded packages are not loaded again.

Other toplevel directives are not yet supported but may be in the future:

#trace function-name	#untrace function-name	#untrace_all

Error Handling

If any of your script commands have syntax errors or raise exceptions (including assert statements), the unified script runner UCramRunner (see OCaml REPL Cram Test Setup) will exit with code 3.

Tip

Consider allowing partial matches with:

  >>> #warnings "-partial-match"

  >>> let (Some found) = List.find_opt
  ...   (String.starts_with ~prefix:"b")
  ...   ["a1"; "b2"; "c3"; "d4"]
  val found : string = "b2"

That is more readable than the full code:

  >>> let found =
  ...   match
  ...     List.find_opt
  ...       (String.starts_with ~prefix:"b")
  ...       ["a1"; "b2"; "c3"; "d4"]
  ...   with
  ...   | Some found -> found
  ...   | None -> (* handle error *)
  ...     invalid_arg "no 'b' found"
  val found : string = "b2"

OCaml REPL Cram Test Setup

1. Run the UCramRunner executable from the mark-in runners table.
   • Pick ``UCramRunner` command line options from either the Automatically Discovered Dune Options (only if Dune is your build tool) or the Manually Configured Options.
2. Optional: Run a renderer that corresponds to your main document

Tip

UCramRunner will set the script filename and the position. If the command line were UCramRunner somefile.u then __FILE__ will be available as somefile.u.

Automatically Discovered Dune Options

1. Run opam install UnifiedScript_Top

2. Create a directory (we'll refer to it as <testdir>) that will hold your .ml.u scripts. You may place them in the same directory as your .ml source code.

3. Run opam exec -- dune ocaml top <srcdir> | opam exec -- UDuneImport [options] <testdir>:

   • REQUIRED: The <srcdir> is the directory containing which *.ml modules you want to test.
   • REQUIRED: The <testdir> is the directory containing your .ml.u scripts from the previous step.
   • RECOMMENDED: There are two important options:
     • --package PACKAGE is the name of the Dune (package) your .ml.u scripts will belong to. In a multi-package Dune project you must set this. If you use the MlModuleRunner you must set this or you may get dependency cycles.
     • --require-project-library PACKAGE1 --require-project-library PACKAGE2 ... are the names of public or private libraries in your Dune workspace that your .ml.u scripts depend on (even transitively).
   • RECOMMENDED: Unless you never will use ocamlformat, add the --disable-ocamlformat option for the reasons given in OCaml Module Script Setup.
   • IMPORTANT: If you want OCaml Module Scripts, use the --ml-glob GLOB_PATTERN to say which files in <testdir> should be built as a OCaml Module Script.
   • A full example is: opam exec -- dune ocaml top ext/MlFront/src/MlFront_Cache/MlFront_Cache | opam exec -- UDuneImport.exe --package MlFront_Cache --ml-glob 'test*.ml' -o ext/MlFront/src/MlFront_Cache/dune-utest.inc --require-project-library MlFront_Core --disable-ocamlformat ext/MlFront/src/MlFront_Cache

4. Add a (include OUTPUT.inc) to the dune file in the <testdir> directory. If the dune file does not exist, create it as an empty file and then add the (include OUTPUT.inc). For the above example, your dune file might look like:

   ; file: ext/MlFront/src/MlFront_Cache/dune
   (include dune-utest.inc)

Manually Configured Options

• UMlModuleRunner only: Will you use ocamlformat ever? Then add the --disable-ocamlformat option for the reasons given in OCaml Modules.

• To load a .cma or .cmo file (that is, to do a #directory "dir-name" followed by a #load "file-name"), use the --load or --load-with-dune command line option. For findlib packages, --load first consults the package META file so same package prerequisites and transitive requires packages are loaded before the requested archive, and a requested archive that is absent from that META is ignored.
  For example, to make the OCaml UnifiedScript_Std library in your Dune workspace with the UCramRunner, the external digestif library, and the unix OCaml library available in a .ml.u script use the dune rules:

  (rule
    (target unit-actual.ml.u)
    (deps unit.ml.u
      ; add the opam packages that contain the findlib
      ; libraries your cram test needs.
    (package UnifiedScript_Std)
    (package digestif))
    (action
    ; load the .cma for each library.
    (run %{bin:UCramRunner} unit.ml.u
          -o %{target} --workspace %{workspace_root}
          --load unix.cma
          --load %{lib:digestif:c/digestif_c.cma}
          --load-with-dune %{cma:../UnifiedScript_Std/UnifiedScript_Std}
    )))
  
  (rule
    (alias runtest)
    (action (diff unit.ml.u unit-actual.ml.u)))

  The --load-with-dune requires relative paths from the <testdir> directory to where the the Dune library (without the .cma extension) would be located in your project tree.

dk Build Scripts

File	Prompt	Multiline	Runs
dk.u or *.dk.u	$	>	Build shell command
	%	end of Lua chunk	Lua command

Caution

dk build scripts are implemented but not yet fully tested.

Three requirements in the dk0 build tool led to the full unified script syntax and the metadata syntax.

• Arbitrary languages: Where cram tests allowed one language (shell) and Mercurial unified tests allowed two languages (shell and Python), there was a need for two different languages (Lua and a variation of PowerShell/POSIX shell) with a third reserved for future use.
• Section headings: There was a need to document and assign meaning to sections of each script. Markdown is commonplace, so ATX headings (lines beginning with one or more "#") were a natural choice for section headings.
• Response metadata: Metadata in the response makes it possible to render the response differently based on the tags. For example, when rendering a unified script in Markdown, the code blocks should be rendered with the correct highlighting language (ex. Lua syntax highlighting for % code blocks). Metadata syntax describes how that is done.

Unified scripts satisfies the three requirements.

## CommonsBase_Build.Apparatus@0.1.0

Bundle a small CMake project in the hello-src/ directory
using the Lua command "unified.asset"
  % unified.asset { name="HelloWorld", dir="hello-src" }
  'asset'
  '790'
  'sha256:847c39531962e987ba69983babf15f244735f3d566d4fdcaa9a94c038484415d'

Run the "run-function" shell command to build HelloWorld
with CMake
  $ run-function CommonsBase_Build.CMake0.F_Build@3.25.3 -d t/o/somewhere/
  > assetmodver=CommonsBase_Build.Apparatus.HelloWorld@0.1.0 assetpath=hw
  > gargs[]=-DCMAKE_BUILD_TYPE=Release
  > bargs[]=--config bargs[]=Release
  > iargs[]=--config iargs[]=Release
  > outexe[]=bin/hello
  \test(pass)
  \dk.object(abi: "Release.Windows_x86", value-id: "oey3zcwqi3bodhkdahazywr4ikitvpthpwsveinsixe3lq7zhcfxq")\;

Main Document Kinds

Plain Text

Filename extension	Notes
.MARKIN.u	MARKIN is the file extension of the mark-in kind

Plain text means there is no rendering post-processing step.

Setup:

1. Run the MARKIN executable from the mark-in runners table. You will not need a renderer.

Markdown

Filename extension	Notes
.md.MARKIN.u	MARKIN is the file extension of the mark-in kind

For Markdown documents, avoid Markdown indented code blocks like:

    This is an indented code block.

Those indented code blocks might be mistaken for unified command prompts.

Instead use fenced code blocks:

```text
This is a fenced code block.
```

Setup:

1. Run the MARKIN executable from the mark-in runners table.
2. Run the U2Markup executable from the mark-in renderers table, which will convert your Markdown unified script into a prettified Markdown.

Important

Unified scripts require that lines starting with <SPACE> <SPACE> <SOME PROMPT> are not present in your main Markdown document. Wrap the shell commands and their output in fenced code blocks; do not use indented code blocks.

OCaml Modules

Filename extension	Notes
.ml

Ordinary OCaml module files can be used without any special markup. However, the toplevel module expressions in the OCaml module file must be formatted. In particular:

• the toplevel module expressions must start at column 1
• the first word (ex. ["let"], ["module"], ["open"], etc.) of each toplevel module expression must be followed by an ASCII space (0x20). No other whitespace is allowed.

ocamlformat conforms to the formatting requirements.

Setup:

1. Run the UMlModuleRunner and options from OCaml Module Script. You will not need a renderer.

Typst

Filename extension	Notes
.typ.MARKIN.u	MARKIN is the file extension of the mark-in kind

Warning

Typst documents have not been tested.

AsciiDoc

Filename extension	Notes
.adoc.MARKIN.u	MARKIN is the file extension of the mark-in kind

Warning

AsciiDoc main documents have not been tested.

Unified Ecosystem

Runners

Executable	Description
UCramRunner	Runs .ml.u scripts with OCaml mark-ins
UMlModuleRunner	Runs .ml scripts with OCaml mark-ins

Renderers

Executable	Description
U2Markdown	Renders commands and responses as Markdown code blocks

OCaml Users

You can install the runners and renderers using opam:

opam install UnifiedScript_Std UnifiedScript_Top

You can also use the bleeding edge:

opam pin add UnifiedScript_Std https://gitlab.com/dkml/build-tools/MlFront/-/releases/permalink/latest/downloads/MlFront.tar.gz

opam pin add UnifiedScript_Top https://gitlab.com/dkml/build-tools/MlFront/-/releases/permalink/latest/downloads/MlFront.tar.gz

# then update periodically with
opam update UnifiedScript_Std UnifiedScript_Top
opam upgrade UnifiedScript_Std UnifiedScript_Top

Supporting a new mark-in kind

Note

TODO: Document how to extend the ecosystem using UnifiedScript_Std