Property Testing

March includes a built-in property testing library inspired by Hedgehog. Instead of writing individual test cases with specific inputs, you describe properties that should hold for all values of a type, and the library generates hundreds of random inputs to verify them.

When a property fails, the library automatically shrinks the failing input to the smallest counterexample that still fails, without any extra work from you.

Quick start

mod MyTests do

  test "addition is commutative" do
    Check.all(Gen.tuple2(Gen.int(-100, 100), Gen.int(-100, 100)), fn pair ->
      match pair do
      (a, b) -> a + b == b + a
      end
    )
  end

end

Run it:

march test my_tests.march

That’s it. Check.all runs the property 100 times with growing input sizes. If it passes, the test passes silently. If it fails, you get a minimal counterexample.

How it works

Generators produce values with built-in shrink trees

Every generator returns not just a value, but a rose tree of shrink candidates. When Gen.int(0, 100) generates 87, the tree looks like:

87
├── 0      (target)
├── 43     (halfway)
├── 65     (three-quarters)
├── 76
├── 81
├── 84
├── 85
└── 86

Each of those nodes has its own children, recursively. This is called integrated shrinking: the shrink strategy is embedded in the generator, not bolted on as a separate step.

Why integrated shrinking matters

When you compose generators with Gen.bind, the shrinking automatically stays coherent. If you generate a list and then an index into that list, shrinking the list won’t produce an invalid index; the index is recomputed from the same random bits as the original.

This is the key advantage over QuickCheck-style property testing, where shrinking is separate and can produce invalid combinations from dependent generators.

Writing properties

A property is a function that takes a generated value and returns Bool:

test "reverse is involutive" do
  Check.all(Gen.list(Gen.int(-50, 50)), fn xs ->
    List.length(List.reverse(List.reverse(xs))) == List.length(xs)
  )
end

Multiple assertions

Use let _ = assert ... to chain multiple assertions. The final expression must be true:

test "sort invariants" do
  Check.all(Gen.list(Gen.int(-100, 100)), fn xs ->
    let sorted = List.sort_by(xs, fn (a, b) -> a < b)
    let _ = assert (List.length(sorted) == List.length(xs))
    let _ = assert (List.all(List.zip(sorted, List.drop(sorted, 1)), fn pair ->
      match pair do (a, b) -> a <= b end
    ))
    true
  )
end

If any assertion fails, the runner catches the failure and shrinks to the minimal input that triggers it.

Crash-catching

Properties that crash (division by zero, match failures, index out of bounds) are also caught and shrunk:

test "safe_divide handles all inputs" do
  Check.all(Gen.tuple2(Gen.int(-100, 100), Gen.int(-100, 100)), fn pair ->
    match pair do
    (a, b) ->
      let _ = safe_divide(a, b)
      true
    end
  )
end

If safe_divide panics for some input, the runner finds the minimal crashing input.

Available generators

Primitives

Strings and characters

Collections

Combinators

Combinators in depth

Gen.map: transform values

-- Generate even numbers
let gen_even = Gen.map(Gen.int(0, 50), fn n -> n * 2)

-- Generate record-like tuples
let gen_point = Gen.map(
  Gen.tuple2(Gen.float(-1.0, 1.0), Gen.float(-1.0, 1.0)),
  fn pair -> match pair do (x, y) -> (x, y) end
)

Gen.bind: dependent generators

When the second generator depends on the first value:

-- Generate a list and a valid index into it
let gen_list_and_index = Gen.bind(
  Gen.filter(Gen.list(Gen.int(0, 100)), fn xs -> List.length(xs) > 0),
  fn xs -> Gen.map(Gen.int(0, List.length(xs) - 1), fn i -> (xs, i))
)

Shrinking the list automatically re-derives a valid index.

Gen.string_of: custom string alphabets

-- Hex strings
let hex_char = Gen.element([48, 49, 50, 51, 52, 53, 54, 55,
                            56, 57, 97, 98, 99, 100, 101, 102])
let gen_hex = Gen.string_of(hex_char)

-- Digit-only strings
let gen_digits = Gen.string_of(Gen.int(48, 57))

Gen.sized: size-dependent generation

The runner grows the size parameter from 0 to 100 across runs. Use Gen.sized to access it:

-- Cap list length at 10 regardless of runner size
let gen_short_list = Gen.sized(fn s ->
  let cap = if s > 10 do 10 else s end
  Gen.list_of_size(cap, Gen.int(0, 100))
)

Configuration

Check.all_with

Override defaults with Check.all_with:

test "stress test with more runs" do
  let config = { Check.default_config() with num_runs: 500, max_size: 200 }
  Check.all_with(Gen.list(Gen.int(0, 100)), fn xs ->
    List.length(xs) >= 0
  , config)
end

Config fields:

Reproducible seeds

When a property fails, the output includes the seed:

property failed after 23 run(s)
  counterexample: 50
  returned false
  shrunk 3 step(s) from: 94
  reproduce with seed: 1234567890

Re-run with the same seed:

march test my_tests.march --seed=1234567890
forge test --seed=1234567890

Skipping property tests

Property tests run hundreds of iterations and can be slow. Skip them for fast feedback:

march test --skip-properties
forge test --skip-properties

All Check.all calls return immediately without running.

Failure output

A failing property produces a report like:

FAIL: "list is always non-empty"
  error: panic: property failed after 1 run(s)
    counterexample: []
    returned false
    shrunk 0 step(s) from: []
    reproduce with seed: 1776290875

For assertion failures, the assertion diagnostic is included:

FAIL: "values are bounded"
  error: panic: property failed after 5 run(s)
    counterexample: 10
    raised: assert 10 < 10
      left:  10
      right: 10
    shrunk 2 step(s) from: 13
    reproduce with seed: 42

For crashes:

FAIL: "no division by zero"
  error: panic: property failed after 21 run(s)
    counterexample: 0
    raised: int_div: division by zero
    shrunk 0 step(s) from: 0
    reproduce with seed: 99

Property patterns

Algebraic laws

Test mathematical properties that must hold:

-- Commutativity: a + b == b + a
-- Associativity: (a + b) + c == a + (b + c)
-- Identity: a + 0 == a
-- Inverse: a - a == 0
-- Distributivity: a * (b + c) == a*b + a*c
-- Idempotence: sort(sort(xs)) == sort(xs)
-- Involution: reverse(reverse(xs)) == xs

Round-trip properties

Encode and decode should cancel:

test "base64 round-trip" do
  Check.all(Gen.lowercase_string(), fn s ->
    let encoded = Base64.encode(Bytes.from_string(s))
    match Base64.decode(encoded) do
    Ok(decoded) -> Bytes.to_string(decoded) == s
    Err(_)      -> false
    end
  )
end

Structural invariants

Operations should preserve certain properties:

-- Length preservation: length(map(f, xs)) == length(xs)
-- Monotonicity: length(filter(p, xs)) <= length(xs)
-- Sorted output: is_sorted(sort(xs))
-- Partition: take(n, xs) ++ drop(n, xs) == xs

Model checking

Compare a complex implementation against a simpler reference:

test "BigInt mul matches Int for small values" do
  Check.all(Gen.tuple2(Gen.int(-100, 100), Gen.int(-100, 100)), fn pair ->
    match pair do (a, b) ->
      BigInt.to_string(BigInt.mul(BigInt.from_int(a), BigInt.from_int(b)))
        == int_to_string(a * b)
    end
  )
end

API reference

Gen module

Primitives: int(lo, hi) (bounds required), int_sized, bool, float, constant, element, ascii_char, lowercase_char

Strings: string, lowercase_string, string_of

Collections: list, list_of_size, option, tuple2, tuple3

Combinators: map, filter, bind, one_of, frequency, sized

Tree inspection (advanced): tree_root, tree_children, tree_singleton, tree_map, tree_bind, tree_filter, run

Check module