Template Testing

Oicana comes with test infrastructure for templates. To get started, create a directory called tests in a template directory. Here is an example test collection tests.toml defining a single snapshot test:

tests.toml

tests_version = 1

[[test]]
name = "with_logo"

[[test.inputs]]
type = "blob"
key = "logo"
file = "../logo.jpg"

[[test.inputs]]
type = "json"
key = "data"
file = "data.json"

All paths in a test collection are relative to its toml file. The collection above defines a test with a blob input and a json input given as logo.jpg in the parent directory and data.json next to the test collection. Executing oicana test for this template, will compile it with those inputs and attempt to compare the output with a with_logo.png living next to the test collection.

Note

On the first run, you’ll need to use oicana test --update to create the initial snapshot files. Subsequent test runs will compare against these snapshots.

Use --update or -u whenever changes in the snapshot files are expected and should be accepted.

The tests directory will be recursively searched for any test collection files in the form of <optional-prefix.>tests.toml.

The prefix is also the collection’s name unless the collection sets one, and the name prefixes the default snapshot file of every test in it. A plain tests.toml has no prefix and therefore no name, so its snapshots are just <test-name>.png:

Collection file	name	Default snapshot for a test called with_logo
tests.toml	(none)	with_logo.png
layout.tests.toml	(none)	layout.with_logo.png
layout.tests.toml	"my_collection"	my_collection.with_logo.png

Example

The example templates table and invoice both define some simple snapshot tests.

PDF standards

Every test also exports the compiled document to PDF using the PDF standards configured in the template manifest. A template that declares standards = ["ua-1"] fails its tests when the document does not meet PDF/UA-1, for example because the document title is missing or an image has no alt text. Since the standards default to ["a-3b"], this check applies to every template.

The failure names the standards and includes the diagnostics from the export:

🔥  Test failures
  -> invoice
  ↳ with_logo
    ↳ Failed to export PDF for the standards 'ua-1' configured in the template manifest:
error: PDF/UA-1 error: missing document title
 = hint: set the title with `set document(title: [...])`

Since the requirements of a standard depend on the content, every test case is checked with its own inputs, including each sample of a fuzzed input.

Set pdf = false to skip the PDF export for a single test or for a whole test collection:

tests.toml

tests_version = 1
pdf = false # no test in this collection exports a PDF

[[test]]
name = "layout_only"

[[test]]
name = "conformance"
pdf = true # ... except this one

Collection settings

snapshot and pdf can be set for a whole test collection. Tests that configure the same setting override their collection:

tests.toml

tests_version = 1
name = "fuzzing" # optional, defaults to the collection file's prefix (empty for a plain tests.toml)
snapshot = false # none of the tests below compare against a snapshot file

[[test]]
name = "fuzz_json_input"

[[test]]
name = "fuzz_other_input"

Watch mode

Pass --watch (or -w) to keep the test runner alive: it executes the suite once, then re-runs the affected tests whenever a source file changes. Useful while iterating on a template. With oicana test --watch you can cover both the “did I break a snapshot?” and “does the manifest still parse?” checks on every save.

JSON input fuzzing

If a JSON input in typst.toml has a schema configured, you can let Oicana fuzz that input as part of a snapshot test.

tests.toml

tests_version = 1

[[test]]
name = "fuzz_json_input"
snapshot = false

[[test.inputs]]
type = "json"
key = "data"
samples = 50

Setting snapshot = false means no image files are created and compared. This is often the right choice for fuzzing tests, because the image output is likely expected to be different for different JSON input values. The configuration of 50 samples will cause Oicana to compile the template with 50 random values for the JSON input that all satisfy the schema. If the schema is very large, it might make sense to increase the number of samples to cover more possible values.

Full example configuration

A maximal and documented example test collection:

tests.toml

tests_version = 1
name = "my_collection" # Optional, default is the collection file's prefix (empty for a plain tests.toml)
pdf = true # Optional, default `true` - whether the tests in this collection export a PDF
# snapshot = false # Optional - default snapshot configuration for all tests in this collection

[[test]]
name = "with_logo" # Required
mode = "development" # Optional, default "production" - decides if `development` values of inputs get used or not
snapshot = "my_snapshot.png" # Optional, default "<collection-name>.<test-name>.png" (just "<test-name>.png" for an unnamed collection) or collection level `snapshot` if set - relative path to a png file that will be compared to the test output or `false` to disable snapshot comparison
pdf = true # Optional, default is collection level `pdf` - export the document with the PDF standards of the template manifest

[[test.inputs]]
type = "blob" # Required - `blob` or `json`
key = "logo" # Required - key of input as configured in the template manifest under test
file = "../logo.jpg" # Required - relative path to a file that will be the value of this input
meta = { image_format = "jpg" } # Optional, default `none` - meta dictionary for the blob input (see input documentation)

[[test.inputs]]
type = "json"
key = "data"
file = "data.json"


[[test]]
name = "test_without_snapshot_comparison"
snapshot = false # this disables comparing the test output with a snapshot file


[[test]]
name = "fuzz_json_input"
snapshot = false

[[test.inputs]]
type = "json"
key = "data"
samples = 50 # this requires that the "data" input has a json schema configured in `typst.toml`


# Any number of additional tests in this collection
[[test]]
name = "a_second_test"