March Docs

WorkDispatch

WorkDispatch: capacity-aware RPC routing (P4 L6).

Routes function calls to the cluster node with the most spare capacity using the power-of-two-choices algorithm. Falls back to local execution when no peer is healthier by DISPATCH_THRESHOLD_MILLI (200 millis = 20% of a core) or when all load readings are stale.

Load readings decay: a NodeLoad sampled_at older than LOAD_STALE_MS (10s) is excluded from routing candidates.

ConsistentHash affinity: dispatch_affine routes by key hash for data-local work (same key always hits the same shard node unless it's down, in which case it falls back to load-aware dispatch).

Functions

fndispatchdispatch(#

Route a call of [fn_ref] with [args] to the best peer in [peers], or run it locally with [invoke_local]:

  1. pick_dispatch_target over [peers] scored by spare CPU; with fewer than
two fresh peers the call runs locally.
  1. Unless the target's spare CPU exceeds [local_spare] by more than 200
milli-cores (a fifth of a core), the call runs locally.
  1. [fd_for] maps the target to its RPC connection; None runs locally.
  2. Otherwise NodeCall.call with deadline now_ms + timeout_ms; a
`NoConnection` or `DeadlineExceeded` falls back to [invoke_local], any
other result (including `TypeMismatch`) is returned as is.

Note that the remote attempt may already have used up the timeout before the local fallback runs.

fndispatch_affinedispatch_affine(#

Consistent-hash affinity dispatch: call [fn_ref] on the ring owner of [key] in [ring] through fd_for(owner_id), so the same key keeps hitting the same node. Falls back to load-aware dispatch (with the same [now_ms]) when the ring is empty, the owner has no connection, or the call fails with NoConnection or DeadlineExceeded; any other result is returned as is.

fnpick_dispatch_targetpick_dispatch_target(#

Pick a dispatch target from [members]: keep the fresh loads (sampled at most 10 s before [now_ms]), then return whichever of the FIRST and LAST fresh entries scores higher under [scorer] (the first on a tie). Returns None when fewer than 2 loads are fresh. The choice is deterministic: for power-of-two-choices, pass a shuffled or randomly sampled [members] list (the transport owns the RNG), or use pick_two.

fnpick_twopick_two(#

Like pick_dispatch_target, but compares the fresh loads at indices [idx_a] and [idx_b] (taken modulo the number of fresh loads), for a caller that has already sampled indices from an RNG. The two indices may name the same entry. Returns None when fewer than 2 loads are fresh, or when an index is negative (the remainder stays negative).