SessionNode

One peer connection: the remote role it reaches and everything that connection owns.

The endpoint actor's state: the delivery count, and the readers' outcomes serve_outcome waits on (see the actor below).

The whole session on this node. handlers holds the installed continuation per local endpoint and waiting the role each expects; pending holds deliveries that arrived before their continuation. meta carries the send seq, the closed flag and the host-gone flag. forward (key "f") is set only for a HOSTED party (run_hosted): a delivery for an awaiting endpoint then goes to the user's actor through it, instead of the installed handler running here, and the actor resumes the endpoint itself (the generated event API).

Failure state: gone maps a peer role that can send nothing more to why ("closed" for a normal close, else the cancellation cause); cancel_hs holds the cancel handler installed with the current continuation (Session.on_cancel); crash_hs the crash continuation installed with it (Session.on_crash), under the endpoint AND the role it is for (crash_key); forward_cancel (key "c") is the hosted party's way to tell its actor that its endpoint was cancelled and forward_crash (key "x") its way to tell it that a role it waits on crashed, so the actor takes the crash branch itself (the crash arrives on the delivery route, as Session.crash_payload, and the generated resume turns it into a Crashed_… event); heard counts heartbeat ticks since a peer's last frame. Our own endpoint's end is in meta: "done" once it closed or was cancelled, and for a cancellation "kind" (see cancel_endpoint), "cancel_role", and why's "cancel_cause".

Where a role listens: its own entry says where; a peer's entry says where to dial it.

Signatures below say List(Addr) outright: an alias does not unify with its expansion across a module boundary, and a caller building the table by hand (Cons(addr, Nil)) must be able to pass it.

Why run ended other than by the protocol reaching its end. A session is not resumable: a peer that is gone ends it for everyone, and run returns rather than hanging on it. What to do next (run again: a fresh party and fresh connections) is the caller's decision.

• Cancelled(role, cause): this role's endpoint was cancelled because

it was waiting on `role`, which can send nothing more, and nothing
from it was queued. `cause` is the chain back to where the failure
began ("role 3: connection lost"). A peer that fails AFTER this role
no longer needs it does not cancel it: `run` returns `Ok`.

• Protocol(role, why): that peer sent what cannot be delivered -- a

message the generated code could not decode or one its state cannot
receive (`Session.fail`). Our endpoint is cancelled and the peers are
told.

• HostGone(ep): a hosted party's actor is gone (crashed or

restarted); its parked session state died with it. The peers are sent
a Cancel.

• Left(why): this role called leave.
• Listen(port, why) / Accept(why) / Connect(role, why): wiring.
• NoOffer(role, why): initiate found no access point that would

take [role] (none registered, all full, all refused, or no answer).

• AlreadyOffered(role): this node already offers [role] of that protocol.
• Unauthorized(role, why): certificate mode (step 11b), and this node's

own certificate does not let it offer [role].

An offer: what an invitation needs to be answered and its session run. [start] runs one session under a sid, abandoned while still forming once its argument says so. [flags] and [strs] are tables SHARED by every offer of the process, keyed under the offer's [id] (its actor's pid): flags "c<id>" (closing: 1, retiring: 2) and "w<id>/<sid>" for a session its initiator withdrew, strs "r<id>" its route key. A Vault table is never freed (the runtime registers it by name for the life of the process), so two per offer were two leaked per offer opened; the keys an offer leaves in the shared ones are dropped when its actor retires (retire_offer). [roles] is the protocol's role-name table (<P>_Msg.role_names()), what the certificate checks name roles by (step 11b).

The fingerprints already in a forming session, besides the initiator's own: at most ONE other (build step 9). [older]: whether it is one the initiator's table accepts (the initiator is the newer side and checks every pair), else an unknown one taken to be newer (each such offer checks the initiator's fingerprint itself, which covers every pair with it, since acceptance is per newer role against the older fingerprint).

Add the peer on a connection this node ACCEPTED. The accepting side announces first, then learns the peer's role from its hello -- a listener does not know which role connects next.

The peers [my_role] accepts: those above it.

The first entry in [addrs] for [role], or None if it has none.

One entry per role from the environment: <PREFIX>_<ROLE>_ADDR = host:port (upper-cased) for each (name, index) in [roles], the generated <P>_Msg.role_names(). An unset variable is no entry, and run names the roles it needed; a set one that does not parse panics. The generated <P>_Run.addrs_from_env() passes the protocol's name and roles.

Stop offering: new invitations are refused and the name is released. Sessions already running finish; the offer's actor ends once none is left (at once when none runs). Closing an offer twice is harmless.

Add the peer in role [their_role] on a connection this node OPENED. The peer announces first; a role in its hello that is not [their_role] is a wiring mistake and fails loudly.

Deliver [msg] from role [from] to local endpoint [ep]: run the continuation installed on [ep] with it, then drain whatever parked messages the new continuation is now willing to take. Returns [ep].

A delivery nobody expects yet (no continuation installed, the installed one waits on a different role, or an older message from the same sender is still parked) is parked, not dropped. If [ep] has already closed or been cancelled the message is discarded, except on a drained endpoint, where it goes back to its sender. Normally called from the party's endpoint actor when a "SessionNode.Deliver" frame arrives; it must run in that actor's turn.

The peers [my_role] dials: those below it, ascending (i < j: i listens, j connects).

Drain every code epoch this process runs, as the reload server's DRAIN epoch:<current> does: every session on this node ends at its next loop boundary (D27; run_<Role> returns Ok(Session.Drained(n))), actors still on an old epoch get a marker at [soft_ms], and at [hard_ms] (> 0) whatever is still pinned is stopped. Topology.drain calls it on SIGTERM. Takes Cap(IO): it ends sessions it does not own.

Tear every connection down: close data, Bye on control, await the control reader, close control, and end the party's Endpoint actor (a cluster-mode party owns no connection: only the actor ends). Then retire the party's tables: call it once serve / serve_outcome has returned.

Start one session of protocol [proto] in role [my_role] on the cluster node [node], filling every role in [others] (all the other roles) from the access points that offer it (offer_role). Mints a fresh session id (this node, its creation, a counter: never reused, so a restarted or partitioned node cannot be addressed by an old session), invites one offer per role -- an offer on this node first, then the others -- moving to the next offer on a refusal or no answer, all within the setup time (MARCH_SESSION_CONNECT_MS, 20 s). Only offers whose version may form a session with this one are invited (build step 9): the same fingerprint, an older one [compat] (the generated <P>_Msg.compat_by_role()) accepts for every role at this fingerprint, or an unknown one that decides for itself; a session holds at most two fingerprints. Then runs [body] as run_cluster does. Err(NoOffer(role, why)) if a role could not be filled; the offers that had accepted are withdrawn. The generated <P>_Run.initiate_<Role> supplies the protocol, its role-name table [roles], the role, peers and other roles.

In certificate mode (step 11b) an offer whose node's certificate does not name "Proto.Role:offer" is skipped before any invitation and listed in NoOffer's reasons; each offer checks this node's certificate names "Proto.Role:initiate" for [my_role] and refuses otherwise; and once formed, every role's endpoint must be on a node certified for that role.

How many sessions [node] is running that no offer counts: those initiate, run_cluster and run_cluster_hosted calls on this node have started and not yet returned from (an offer's sessions are its OfferCount.active). Topology's SIGTERM drain waits for both.

The roles [my_role] needs an address for and [addrs] lacks: every peer it dials, and itself if it accepts anyone. Empty means run can start.

The fingerprint in an offer's registry name [name] (after [prefix], the protocol and role part), or "" for a name in the older form ap:<P>/<role>/<node>, which carried none.

offer_role, with the role HOSTED IN AN ACTOR [host] (its pid as an Int) for every session at once: each accepted session runs run_cluster_hosted in its own task, and the actor keeps one parked endpoint per session id (a LinearMap), which is why every callback carries the session id: start(sid, s), deliver(sid, s, from, msg, ep), cancel(sid, s, role, cause, ep). The host is monitored once per session; if it dies, every session it hosts ends with HostGone. The generated <P>_Run.offer_hosted_<Role> supplies the protocol, role and peers.

Offer role [my_role] of protocol [proto] (fingerprint [fingerprint], the generated <P>_Msg.fingerprint()) on the cluster node [node], for up to [capacity] sessions at once. Each session forms when an initiator (initiate) invites this node; it then runs [body] in its own task, as run_cluster would, under the session id the initiator minted. An invitation is refused when the offer is full, closing, or built from a version of the protocol this one cannot form a session with (build step 9: the same fingerprint, or one the compatibility table [compat], the generated <P>_Msg.compat_by_role(), relates to it), and, in certificate mode, when the initiating node's certificate does not name "Proto.Role:initiate" for the role it plays (verdict "initiator <node> not authorized for Proto.Role"). In certificate mode this node's own certificate must name "Proto.Role:offer" for [my_role], else Err(Unauthorized(..)). [roles] is the protocol's role-name table (step 11b). One offer per role and fingerprint per node: a second is Err; two versions of one role may be offered side by side. The generated <P>_Run.offer_<Role> supplies the protocol, role and peers. The offer's name is bound to an actor of this node; if the node stops, or a supervisor restarts the program that made it, it is gone from the registry and an initiator no longer picks it -- offer again to be found again.

The two-party case in one call: a party of one peer over [conn], with role numbers left at 0 (a single-peer party routes every message to its one connection). [accepted] says which side of the connection this is.

The Session.Ops for [p]: hand it to Session.attach.

Parse host:port into the entry for [role].

Start a session on this node in role [my_role] (<P>_Msg.role_<R>()), with no peers yet: add one per remote role with accept_from / connect_to. [on_close] runs when a local endpoint closes, with its endpoint id.

From now on, every initiate on [node] is refused with Err(Left("draining")) before it invites anyone: the node is draining (Topology's SIGTERM drain calls it), so a task that keeps starting sessions -- a pool hook's traffic loop -- cannot keep the drain from ever seeing zero sessions. Sessions already started run on. Permanent for the node.

Has refuse_initiations been called for [node]?

Check that [p] has a connection to every role in [peers] -- the generated <P>_Msg.peers_<R>(), every role [R] exchanges a message with -- and panic naming the missing ones. Call it before serve, so a wiring mistake fails at startup rather than at the first emit to an unreachable role. A party with exactly one peer passes: it routes everything to that connection whatever the role numbers say (the two-party case, where open never asked for them).

Run role [my_role] of a session on this node: listen if any peer is above it, connect to every peer below it (ascending, with retry), accept every peer above it, check the party against [peers] (<P>_Msg.peers_<R>()), attach, run [body] to its first suspension, serve, and tear down. [addrs] must hold this role's own entry iff it accepts anyone and an entry for every peer it dials; a missing one panics by role before any socket opens. [node_id] names this node in the handshake and [secret] is the cluster secret every node shares; with MARCH_NODE_CERT set the node runs in certificate mode instead (ClusterNode.auth_from_env, the variables a cluster node reads), and each peer's certificate must let it play the role it announces in [proto] (named by [roles], <P>_Msg.role_names()), else setup fails with Connect/Accept "role P.R is played by <node>: not authorized for P.R". [fingerprint] is the protocol's own (<P>_Msg.fingerprint()): it rides the hello, and a peer whose own differs -- or that is too old to send one at all -- is refused at setup with a Connect/Accept error rather than discovered mid-session as an undecodable message (2026-09-21). "" asks for no check. The generated <P>_Run.run_<Role> supplies the role, its peers, the fingerprint and the typed entry state.

Ok(Session.Finished) when every peer's endpoint closed, Ok(Session.Drained(n)) when the session was ended at a loop boundary by a drain (D27; [n] of this role's messages came back undelivered); else the RunError that ended the session, after the party has been torn down.

run, over the running cluster node [node] (ClusterNode.start) instead of connections of its own: role [my_role] of session [session] (any string the roles agree on -- a fresh one per session). Each role registers its endpoint under the session's name and finds its peers' by name (waiting up to 30 s), then [body] drives it exactly as for run. A peer is gone when it closes or is cancelled, or when its node's data connection ends -- SWIM declared the node dead ("node <name> dead: <cause>") or the connection was lost. Every role must be on a different node.

[fingerprint] is the protocol's (<P>_Msg.role_fingerprint(role)): each role registers it beside its endpoint, and a peer that registered a different one, or none (a node built before the check), is refused with Err(Connect(role, why)) before the session forms, as the direct runner refuses it at the hello. "" checks nothing. The generated <P>_Run.cluster_<Role> supplies the role, its peers and the fingerprint.

run_cluster, with the role HOSTED IN AN ACTOR [host] (its pid as an Int), as run_hosted_or hosts it over the runner's own connections: [start] gets the session capability once the party is formed, [deliver] every delivery the actor's parked endpoint awaits, (s, from, msg, ep), and [cancel] a cancellation, (s, role, cause, ep). The host is monitored: its Down ends the session with Err(HostGone(ep)). The generated <P>_Run.cluster_hosted_<Role> supplies the role and its peers and adds the session id to every callback. [fingerprint] is checked as for run_cluster.

A one-line, human-readable description of [e], prefixed "session_node: ", with roles shown as their numbers (see run_error_message_named for role names). For example Connect(2, "refused") gives "session_node: connect to role 2: refused".

run_error_message with roles shown by [name] (the generated <P>_Msg.role_name): what <P>_Run.error_message is.

run, with the role HOSTED IN AN ACTOR [host] (its pid as an Int, pid_to_int): the generated event API (Parked_<Role> in the actor's state, await_*/resume) instead of a body. [start] runs once the party is connected, with the session capability: send it to the actor, whose start handler registers and parks. [deliver] is called for every delivery the actor's parked endpoint awaits -- (s, from, msg, ep), what its resume handler takes: send them on. Deliveries are one per suspension, in mailbox order, and one that arrives before the host has re-parked is held for it.

The host is monitored: its Down (a crash, a kill, a supervisor's restart) cancels its role with Err(HostGone(ep)), and the peers are sent a Cancel (they are cancelled in turn only if they were waiting on it). A restarted actor starts Idle and cannot continue a session its predecessor parked; run again for a new one. run_hosted_or also tells the actor itself. The generated <P>_Run.host_<Role> supplies the role and its peers.

run_hosted, telling the actor when its endpoint is cancelled: [cancel] gets (s, role, cause, ep) -- the role it was waiting on and why that role is gone. Send it to the actor, whose handler stores the generated <P>_<Role>.cancel(state.parked) (a Closed parked value) and may update the rest of its state; it cannot communicate in the cancelled session.

serve_outcome, panicking if our endpoint was cancelled; the transport's original contract.

Read deliveries from every peer until each connection has ended: the peer closed, was cancelled, or went away. A peer's failure no longer ends the session for this node at once: it cancels our endpoint only if our endpoint is waiting on that peer with nothing from it queued (Maty's rule; see the module comment), and cancellation then spreads to our other peers as a Cancel frame. Nil if our endpoint ended normally, else (role, cause) for its cancellation. Every resumption runs in this node's endpoint actor, so call it after driving the local role to its first suspension. serve panics on a cancellation; run returns it.