March Docs

Topology

Topology: the runtime side of a topology app (build step 3 of the distributed-deploys plan, specs/plans/2026-09-21-distributed-authority-and-deploys-plan.md, sections 4.1, 4.2 and II.3 / II.6 "Placement at runtime").

In a topology app main is generated (march --topology, lib/desugar/desugar_topology.ml). It starts the ClusterNode, runs each pool's hook under hook (a watchdog: a hook must return promptly), and hands this module one Role per served role. From then on the NODE owns its offers (D16): place opens the offers the topology places on it and re-evaluates on every membership change; drain_on_signal closes them on SIGTERM/SIGINT and lets running sessions finish; supervise_offers re-offers a role whose offer (or hosting actor) died, and keeps main alive.

Placement rules (4.2), evaluated by every node for itself, with no control plane: everywhere() offered on every node that serves the role (D18); on_label(label) offered when this node's labels (MARCH_NODE_LABELS, a comma-separated list set by the launcher) include it; count(n) offered on n of the eligible live nodes, chosen by rendezvous hashing: every node ranks the candidates by hash(role, node_id) and offers when it is in the top n (D19); count_on(l, n) count(n) among the nodes carrying label l. "Eligible" for a Count role means: a live member (or this node) that serves the role and carries its label. Other nodes' pools and labels are not in the membership table, so each eligible node registers a cluster name, topo:<role>/<node_id>, and the ranking reads those names back; a dead node's names become invisible with its membership. A node that REJOINS counts only after MARCH_PLACEMENT_SETTLE_MS (default 15000), so a flapping node does not pull a role back and forth (hysteresis). count(1) is not a lock: during a partition each side may offer, as with any global name.

The hash is SHA-256 (Crypto.sha256), not the plan's BLAKE3: the runtime's BLAKE3 is not exposed to March code, and the placement only needs a hash that every node computes identically.

Losing a role is a drain, not a cut: SessionNode.close_offer refuses new invitations and lets the sessions already running finish.

Re-reading the topology (D16, build step 8). place installs a SIGHUP watcher: on SIGHUP the node re-reads MARCH_TOPOLOGY_FILE (the digest, .forge/topology.json, which the reconciler rewrites before signalling) and applies what changed for the roles its build contains: a new placement, a role this process's pools no longer serve (its offer is closed: a drain), a new capacity (the offer is closed and reopened at the new size). The next evaluation opens and closes offers as for a membership change; nothing restarts. A role newly served that this build has no code for is reported, not offered: that needs a rebuild. The node reports what it applied and the offers it holds to MARCH_TOPOLOGY_STATUS (forge topology status reads it). The reconciler never opens or closes an offer itself.

Types

typeTopoPlacementTopoPlacement = PlaceEverywhere | PlaceOn(String) | PlaceCount(Int) | PlaceCountOn(String, Int) | PlaceNowhere#
typeTopoOpenedTopoOpened = { ap : SessionNode.Offer, alive : Int -> Bool, capacity : Int, stop : Int -> () }#
typeTopoRoleTopoRole = { name : String, place : TopoPlacement, capacity : Int, pinned : Bool,#
typeTopoAnchorStateTopoAnchorState = { topology_anchor_marks : Int }#
typeTopoStateTopoState = { meta : Vault(Int), strs : Vault(String), offers : Vault(TopoOpened), draining : Vault(TopoOpened),#
typeTopoDesiredTopoDesired = { d_name : String, d_place : TopoPlacement, d_capacity : Int }#

Functions

fnactor_roleactor_role(name : String, place : TopoPlacement, mk : () -> Pid(a),#

A role bound to an actor (actor = "M.A" in [roles], D23): one actor per offer, spawned by [mk] (the generated fn () -> spawn(M.A, env)) and handed to [open] (the generated <P>_Run.offer_hosted_<R> call whose start, deliver and cancel callbacks forward to it). An actor spawned for an offer that could not be opened is kept for the next attempt rather than spawned again. Supervision re-offers the role when its actor dies. When the offer closes (placement moved the role, its capacity changed, or its protocol did: reoffer), the actor keeps serving the sessions it hosts and is stopped once they have all ended (6.1: a protocol change spawns a new hosting actor; the old one drains and exits).

fnapply_desiredapply_desired(base : List(TopoRole), desired : List(TopoDesired)) : List(TopoRole)#

[base] (the roles this build contains, as the generated main placed them) under [desired]: a pinned role is left as the build placed it; a role the digest no longer serves here is PlaceNowhere; otherwise its placement and (unless fixed by the build, capacity 0) its capacity come from the digest.

fnclose_allclose_all(node : Cap(ClusterNode.Live)) : ()#

Stop placing and close every offer of [node] (running sessions finish): what SIGTERM does under drain_on_signal. Returns at once.

fnconfig_from_envconfig_from_env() : ClusterNode.CnConfig#

The node's configuration from the environment (ClusterNode.config_from_env), with defaults for a lone local run: MARCH_NODE_NAME defaults to local-1 and MARCH_NODE_PORT to 7946.

fncountcount(n : Int) : TopoPlacement do PlaceCount(n) end#

Offer the role on [n] of the eligible live nodes, by rendezvous hashing (place = { count = n }).

fncount_ofcount_of(p : TopoPlacement) : Int#

How many nodes [p] places on (n for a count = n role; 0 for an unranked one).

fncount_oncount_on(label : String, n : Int) : TopoPlacement do PlaceCountOn(label, n) end#

count(n) among the nodes carrying [label] (place = { on = label, count = n }).

fndesired_ofdesired_of(text : String, pools : String -> Bool) : Result((List(TopoDesired), Int, Int), String)#

What the digest [text] asks of this process: for each role one of [pools] serves, its placement and capacity (a role no such pool serves is absent); the drain deadlines; and an error for text that is not a version-1 digest.

fndraindrain(node : Cap(ClusterNode.Live), soft_ms : Int, hard_ms : Int) : ()#

Drain [node] now, as SIGTERM does under drain_on_signal.

fndrain_on_signaldrain_on_signal(node : Cap(ClusterNode.Live), soft_ms : Int, hard_ms : Int) : ()#

On SIGTERM or SIGINT: close every offer of [node] (new invitations are refused), drain every code epoch the process runs (running sessions end at their next loop boundary, D27, and actors take the epoch drain's deadlines), then exit 0 once no session is running. At [soft_ms] a drain still waiting is reported; at [hard_ms] the process exits 1, naming how many sessions it cut. A second signal escapes the watcher and stops the process at once.

fneligible_hereeligible_here(p : TopoPlacement, labels : List(String)) : Bool#

Whether this node may take part in [p] at all: it carries the label, if any.

fneverywhereeverywhere() : TopoPlacement do PlaceEverywhere end#

Offer the role on every node that serves it (no place in the topology).

fnhookhook(name : String, f : () -> a) : a#

Run the pool hook [name] ([f]), which must return promptly: offers open only after every hook has returned. A watchdog reports a hook still running after MARCH_HOOK_TIMEOUT_MS (default 10000) and exits the process with status 1: offering nothing would be worse than failing loudly. Long-lived work belongs in a task or actor the hook spawns.

fnlabels_oflabels_of(s : String) : List(String)#

Labels from a comma-separated list: `\

fnmarkedmarked(role_name : String, names : List(String)) : List(String)#
fnmarkermarker(role_name : String, node_id : String) : String#

The name an eligible node registers for a ranked role.

fnmay_openmay_open(p : TopoPlacement, others : Int, now : Int, started : Int, synced : Bool,#

Whether this node may OPEN ranked role [p] it now ranks for (it is not offering it yet), at [now]: the settle rule its peers apply to it, applied by the node itself. A node that started at [started] and has not yet merged a peer's registry ([synced] false) knows no other node's marker, so it opens nothing ranked until it has, or until [settle_ms] have passed (seeds unreachable: a lone node still takes its roles). A node that a peer last showed as rejoining at [rejoined_at] (ClusterNode.rejoined_at) is not counted by those peers for [settle_ms], so for that long it opens the role only if the [others] other non-dead nodes marked eligible cannot fill its count without it. Never stops a node keeping an offer it holds.

fnnode_labelsnode_labels() : List(String) do labels_of(Env.get("MARCH_NODE_LABELS", "")) end#

This node's labels: MARCH_NODE_LABELS, comma-separated (set by the launcher from the overlay).

fnnode_namenode_name(node : Cap(ClusterNode.Live)) : String do label(node) end#

This node's name as the membership and the control plane know it.

fnoffer_actor_roleoffer_actor_role(name : String, place : TopoPlacement, capacity : Int, mk : () -> Pid(a),#

actor_role, offered at [capacity] sessions: [open] takes the actor and the capacity (the generated fn (topology_a, topology_cap) -> <P>_Run.offer_hosted_<R>(io, node, topology_cap, topology_a, ...)).

fnoffer_roleoffer_role(name : String, place : TopoPlacement, capacity : Int,#

A role bound to a function, offered at [capacity] sessions: [open] is the generated fn topology_cap -> <P>_Run.offer_<R>(io, node, topology_cap, ...). A reload that changes the role's capacity reopens its offer at the new size.

fnoffer_workeroffer_worker(node : Cap(ClusterNode.Live), role_name : String) : Option(Pid(SessionNode.OfferCount))#

The actor answering invitations for [role_name]'s open offer on [node], if one is open.

fnofferedoffered(node : Cap(ClusterNode.Live)) : List(String)#

The roles [node] offers right now, by name, sorted.

fnofferingoffering(node : Cap(ClusterNode.Live), role_name : String) : Bool#

Does [node] offer [role_name] right now? (For a count = 1 role: is this node the one that serves it.)

fnon_labelon_label(label : String) : TopoPlacement do PlaceOn(label) end#

Offer the role on the nodes whose MARCH_NODE_LABELS include [label] (place = { on = label }).

fnpinnedpinned(r : TopoRole) : TopoRole#

[r], pinned: a pushed topology leaves its placement and capacity as the build set them (apply_desired). For roles the generated main adds itself and a topology digest cannot name, such as the control plane's Ctl.Control (placed from [control]): without the pin, every re-read of the topology (a signed push, SIGHUP, or a restart onto a verified copy) would place it nowhere, and the cluster would lose its leader.

fnplaceplace(node : Cap(ClusterNode.Live), roles : List(TopoRole)) : ()#

Open the offers the topology places on [node], one per entry of [roles], and keep them placed: a task re-evaluates every MARCH_PLACEMENT_TICK_MS (default 200 ms) against the current membership, opening a role this node now ranks for and closing (draining) one it no longer does. Membership events are recorded as they arrive; a node that rejoins counts after the settling period. This node applies that period to itself too (may_open): it opens no ranked role before its first registry sync (bounded by MARCH_PLACEMENT_SETTLE_MS), and while a peer counts it as rejoined it leaves a ranked role to the nodes already offering it. Returns once the first evaluation has opened the offers it may open now (a ranked one held back opens on a later tick).

fnplacement_of_jsonplacement_of_json(j : Option(Json.JsonValue)) : TopoPlacement#

A digest's place object as a placement: null is everywhere (D18), { on }, { count }, or both.

fnplacement_textplacement_text(p : TopoPlacement) : String#

A placement as the topology writes it, for reports.

fnrankrank(role_name : String, ids : List(String)) : List(String)#

[ids] ranked for [role_name], best first (highest score; ties by id).

fnrankedranked(p : TopoPlacement) : Bool#

Whether [p] ranks nodes (and so needs the eligible set).

fnreconcile_nowreconcile_now(node : Cap(ClusterNode.Live)) : () do reconcile(node, state_of(node)) end#

Re-evaluate [node]'s placement now, as the next tick would. For tests that drive membership through a fake dictionary and should not wait for a tick (run them with a long MARCH_PLACEMENT_TICK_MS).

fnreloadreload(node : Cap(ClusterNode.Live)) : ()#

Re-read the topology now, as SIGHUP does: the signature-checked copy the reload server keeps (MARCH_TOPOLOGY_VERIFIED_FILE) when there is one, else MARCH_TOPOLOGY_FILE, is parsed and the roles this build contains take their placement and capacity from it; the next evaluation (run at once) opens and drains offers to match. A file that cannot be read or parsed changes nothing and is reported.

fnreofferreoffer(node : Cap(ClusterNode.Live)) : () do reoffer_with(node, state_of(node)) end#

Re-open each of [node]'s open offers with the code now running (distributed-deploys build step 9): an offer whose protocol fingerprint changed is opened again under the new one (a hosted role with a fresh actor) and the old offer is closed; its sessions finish on the old version and a hosting actor exits once they have (6.1). An offer whose fingerprint did not change stays as it is (the new offer's name is already taken). The placement loop runs this itself after a deploy; call it directly only from a test.

fnrolerole(name : String, place : TopoPlacement, open : () -> Result(SessionNode.Offer, SessionNode.RunError)) : TopoRole#

A role bound to a function (body = "M.f" in [roles]). [open] is the generated <P>_Run.offer_<R>(io, node, capacity, fn (s, caps..., st) -> M.f(env, s, caps..., st)).

fnrole_candidatesrole_candidates(node : Cap(ClusterNode.Live), role_name : String) : List(Membership.Member)#

The members that currently mark themselves eligible for the ranked role [role_name] (a place = { on = ..., count = n } role), this node included when it is: its candidates, whichever of them ranks first. The control plane finds its standbys and its leader with it. A dead member is left out.

fnrunningrunning(node : Cap(ClusterNode.Live)) : Int#

Sessions still running on [node]'s offers, open or draining.

fnruns_poolruns_pool(pool : String) : Bool#

Whether this process runs pool [pool]: MARCH_POOLS (comma-separated) names the pools a process of a shared build runs; unset, it runs every pool the build contains (forge run's one process, D9).

fnscorescore(role_name : String, node_id : String) : String#

A node's rendezvous score for [role]: every node computes the same one.

fnsettledsettled(now : Int, rejoined_at : Option(Int), settle_ms : Int) : Bool#

A member that rejoined at [rejoined_at] counts again once [settle_ms] have passed.

fnstart_failedstart_failed(why : String) : a#

Report that the node could not start ([why]) and stop the program with status 1. The generated main starts the node itself -- ClusterNode.start(io, Topology.config_from_env()) -- because only ClusterNode may hand out a Cap(ClusterNode.Live).

fnstatus_textstatus_text(node : Cap(ClusterNode.Live)) : String#

What this node reports to the reconciler: its name, the digest it applied (sha256, or compiled before any reload), its offers, how many closed offers are still draining, the sessions running on them all, and a conflict line per placed role that cannot be offered because another offer on this node holds its endpoint name.

fnstopstop(node : Cap(ClusterNode.Live)) : ()#

Stop [node] and everything placed on it: close every offer (running sessions are cut when the node goes), end placement and supervise_offers, then ClusterNode.stop. For a program (a test, say) that ends on its own rather than on a signal.

fnsupervise_offerssupervise_offers(node : Cap(ClusterNode.Live)) : ()#

Re-offer any role whose offer actor, or hosting actor, has died, and keep the calling task (the generated main) alive until the node stops or drains. A process whose main returned would exit, taking its offers with it.

fnwantswants(p : TopoPlacement, role_name : String, me : String, labels : List(String), eligible : List(String)) : Bool#

Whether node [me], with [labels], offers [role_name] under [p], given the settled eligible nodes [eligible] (other nodes; [me] is added when it is eligible itself).