Cluster over TCPExperimental

Actors across machines.

Swoole TCP mesh between Nexus nodes — gossip membership, phi-accrual failure detection, and location-transparentClusterRef. The sametell() and ask()you already use, whether the target actor lives in the same process or on another machine. Experimental and pre-1.0: the wire protocol and APIs may change, and it is not yet production-hardened.

Clustering guide Package reference

bootstrap.php

<?php
use Monadial\Nexus\Cluster\NodeAddress;
use Monadial\Nexus\Cluster\Tcp\ClusterNode;
use Monadial\Nexus\Cluster\Tcp\ClusterTopology;
use Monadial\Nexus\Cluster\Tcp\NodeEndpoint;
use Monadial\Nexus\Runtime\Swoole\SwooleRuntime;
use Monadial\Nexus\Serialization\TypeRegistry;

$runtime  = new SwooleRuntime();
$system   = ActorSystem::create('my-cluster', $runtime);

$topology = ClusterTopology::create(
    clusterName:       'production',
    self:              new NodeAddress('production', 'eu', 'orders', 'node-1'),
    bindEndpoint:      NodeEndpoint::fromString('0.0.0.0:7361'),
    advertiseEndpoint: NodeEndpoint::fromString('10.0.0.1:7361'),
    seeds:             [NodeEndpoint::fromString('10.0.0.2:7361')],
);

$registry = new TypeRegistry();
$registry->registerFromAttribute(OrderPlaced::class);

$node = ClusterNode::boot($system, $topology, $registry);
$node->expose($processorRef);           // register actor for remote delivery

// Location-transparent: same tell() whether the target is local or remote.
$remoteRef = $node->refFor(
    new NodeAddress('production', 'eu', 'orders', 'node-2'),
    $processorRef->path(),
);
$remoteRef->tell(new OrderPlaced($orderId));

$system->run(); // starts event loop; gossip and failure detection are self-driving

Self-joining TCP mesh.

Configure a list of seed endpoints. On boot,ClusterNode dials each seed, exchanges a Handshake frame, and receives aHandshakeAck containing the full endpoint view of the current cluster. A new node needs to know only one live peer to join — gossip propagates the rest within one interval.

→bindEndpoint — the address the TCP server listens on (e.g. 0.0.0.0:7361)
→advertiseEndpoint — what peers connect to (pod IP in Kubernetes, public IP on bare-metal)
→Gossip and heartbeat default to 1 s; the full view propagates within one round
→Auto-reconnect with exponential back-off (100 ms → 30 s) on TCP disconnection
→singleNode: true starts without seeds — useful for single-node dev and integration tests

bootstrap.php — topology

<?php
// bindEndpoint   — the address the TCP server listens on (can be 0.0.0.0).
// advertiseEndpoint — what peers use to connect back (pod IP in Kubernetes,
//                     public IP on bare-metal). May be identical on bare-metal.

$topology = ClusterTopology::create(
    clusterName:       'production',
    self:              new NodeAddress('production', 'eu', 'orders', 'node-1'),
    bindEndpoint:      NodeEndpoint::fromString('0.0.0.0:7361'),
    advertiseEndpoint: NodeEndpoint::fromString('10.0.0.1:7361'),
    seeds:             [
        NodeEndpoint::fromString('10.0.0.2:7361'), // node-2
        NodeEndpoint::fromString('10.0.0.3:7361'), // node-3
    ],
    gossipInterval:    Duration::seconds(1),        // default
    heartbeatInterval: Duration::seconds(1),        // default
);

// On boot, ClusterNode dials each seed and exchanges a Handshake frame.
// The HandshakeAck carries a full endpoint view of the current cluster —
// endpoint discovery completes within one gossip round, even without
// direct links between every pair of nodes.

src/Actor/OrderActor.php

<?php
// refFor() returns ClusterRef<T> — a full ActorRef<T> implementation.
// The node auto-selects local delivery or TCP at runtime.

$remoteRef = $node->refFor(
    new NodeAddress('production', 'eu', 'orders', 'node-2'),
    $processorRef->path(), // ActorPath agreed by naming convention
);

// tell() — fire-and-forget; self-node calls go directly to the mailbox
$remoteRef->tell(new OrderPlaced($orderId));

// ask() — request-response with timeout; returns Future<R>
/** @var OrderConfirmed $reply */
$reply = $remoteRef->ask(new PlaceOrder($orderId), Duration::seconds(5))->await();

// Self-node short-circuit: if $remoteRef points to this node, the message
// goes directly to the local actor mailbox — no TCP frame, no serialiser.
// The calling code is identical in both cases.

Location-transparent actors.

ClusterRef<T> implementsActorRef<T> — the same interface as every local actor reference. Your handler calls tell()or ask() and never knows whether the target lives in the same process or on another machine. Swap a local ref for a cluster ref without touching a single handler.

→tell() is fire-and-forget; self-node calls bypass TCP entirely — direct mailbox delivery
→ask() stamps a correlation ID and reply path, returns Future<R>
→Messages serialised with MessagePack via nexus-serialization-msgpack; types require #[MessageType]
→AskTimeoutException on timeout; AskCapacityExceededException when the registry is full (fail-fast)

ClusterRef reference →

Phi-accrual failure detection.

Gossip frames double as heartbeats — there is no separate ping/pong. The Hayashibara phi-accrual detector tracks inter-arrival times and computes a continuous suspicion value. When phi exceeds the threshold the node becomes Suspect; when it exceeds it substantially — or gossip silence exceedsmaxNoHeartbeat — the node is markedDown.

→phiThreshold: 8.0 default — conservative, Hazelcast-parity; lower = more sensitive
→sampleSize: 200 — statistical history window of inter-arrival samples
→minStdDev: 500 ms — noise floor preventing phi spikes on idle connections
→TCP EOF → immediate Suspect(Connection) before phi fires
→Graceful shutdown() → immediate Down via Leave frame; no phi wait
→All four knobs tunable via withFailureDetection()

bootstrap.php — failure detection

<?php
// Hayashibara phi-accrual failure detector — the same algorithm used by
// Hazelcast and Akka. Gossip frames double as heartbeats; inter-arrival
// times build a statistical window that drives the phi value.
//
// phi >= phiThreshold → Suspect. phi >> threshold or silence beyond
// maxNoHeartbeat → Down. TCP EOF → immediate Suspect(Connection).
// Graceful shutdown() → immediate Down via Leave frame, no phi wait.

$topology = ClusterTopology::create(/* ... */)->withFailureDetection(
    sampleSize:     200,                    // history window (inter-arrival samples)
    minStdDev:      Duration::millis(500),  // noise floor — prevents spikes on idle links
    maxNoHeartbeat: Duration::seconds(10),  // hard bound: Down if silent for 10 s
    phiThreshold:   8.0,                    // default — conservative; Hazelcast-parity
);

// PSR-14 events fired on state transitions:
//   NodeUp(node, endpoint)
//   NodeSuspected(node, reason: Connection | Gossip | Phi)
//   NodeDown(node)

bootstrap.php — TLS

<?php
// Optional Swoole SSL — encrypts every peer connection, inbound and outbound.
// Never expose plaintext cluster ports to untrusted networks.

$tls = new TlsConfig(
    certFile:   '/certs/node.crt',
    keyFile:    '/certs/node.key',
    caFile:     '/certs/ca.crt',  // null -> no CA verification (dev only)
    verifyPeer: true,              // mutual TLS — recommended for production
);

$topology = ClusterTopology::create(
    clusterName:       'production',
    self:              new NodeAddress('production', 'eu', 'orders', 'node-1'),
    bindEndpoint:      NodeEndpoint::fromString('0.0.0.0:7361'),
    advertiseEndpoint: NodeEndpoint::fromString('10.0.0.1:7361'),
    seeds:             [NodeEndpoint::fromString('10.0.0.2:7361')],
)->withTls($tls);

// Transport selection is automatic. No code changes are needed beyond
// setting TlsConfig — SwooleMeshTransport applies it to all connections.

TLS for encrypted peer traffic.

Pass a TlsConfig toClusterTopology::withTls() and every peer connection — inbound and outbound — uses Swoole SSL. Certificate, key, and an optional CA file are the only required fields. Enable mutual TLS withverifyPeer: true (the default).

→The mesh is open by default — TLS encrypts the wire, but admission control needs a shared secret: withAuthSecret() enables HMAC handshake authentication so only nodes holding the secret can join. Set one in production
→Applied uniformly: SwooleMeshTransport reads TlsConfig for both server and outbound client sockets
→caFile: null skips CA verification — acceptable for dev, never for production
→No code changes beyond topology config; transport selection remains automatic
→Inbound links are capped and unfinished handshakes time out — bounded against trivial resource exhaustion (withInboundLimits())
→Never expose plaintext cluster ports (default 7361) to untrusted networks

One trace. Two nodes. One span tree.

Pass any Observability instance toClusterNode::boot(). W3Ctraceparent is injected into everytell() and ask()frame — the receiving node extracts it and opens a child span automatically. The full call chain across machines appears as a single distributed trace in your collector, with no manual wiring.

→cluster.send and cluster.ask Producer spans around every outbound frame
→cluster.receive Consumer span on the receiving node, parented to the sender's trace context
→cluster.handshake Internal span per peer connection
→Six counters + one bytes-sent histogram — automatically instrumented
→Disabled by default: omit $obs and telemetry costs nothing at runtime

Observability wiring →

bootstrap.php — observability

<?php
// Pass any Observability instance to ClusterNode::boot().
// Omit it (or pass null) for zero-overhead NoopObservability.
$node = ClusterNode::boot($system, $topology, $registry, observability: $obs);

// W3C traceparent is injected into every tell() / ask() frame.
// One trace ID follows the call across machines — no manual wiring.
//
// Example: HTTP on Node A -> TCP -> actor on Node B (one trace):
//
//   POST /orders              [120ms]   (Node A)
//   L orders.handle           [110ms]
//      L cluster.send           [3ms]   (TCP to Node B, Producer span)
//         L order-processor    [95ms]   (Node B — child span, same trace)

// Metrics emitted automatically per ClusterNode:
// nexus.cluster.messages.sent               — remote tells dispatched
// nexus.cluster.messages.local_shortcircuit — self-node tells (no wire)
// nexus.cluster.asks.sent                   — remote asks dispatched
// nexus.cluster.asks.capacity_rejected      — registry-full rejections
// nexus.cluster.bytes.sent                  — outbound frame size (histogram)
// nexus.cluster.frames.sent                 — total frames (all types)
// nexus.cluster.handshake.rejected          — parse-failure rejections

PSR-14 membership events.

Pass any EventDispatcherInterface to theActorSystem and every membership transition dispatches a typed, immutable event — no polling required. Wire them to your metrics pipeline, alerting rules, or audit log.

Node lifecycle

NodeUp(node, endpoint) fires when a peer is confirmed reachable after a Handshake.NodeDown(node) fires when the detector declares the node dead or a Leave frame is received.

Suspicion

NodeSuspected(node, reason) fires when phi exceeds the threshold or a TCP EOF is detected before phi fires.reason is one ofConnection,Gossip, orPhi — precise enough for differentiated alerting.

Transport lifecycle

PeerConnected(peer, endpoint) fires after a successful Handshake. PeerDisconnected(peer)fires on TCP close — before phi, for fast failure propagation.

Quorum

ClusterDegraded(reachableMembers, requiredMembers)fires while the node is below the withMinimumMembers()floor — the quorum-loss signal to alert on.

What the cluster is not.

nexus-cluster-tcp is a best-effort AP transport — available under partition, not consistent. It provides gossip-based membership and phi-accrual failure detection, not a consensus layer.

×No leader election; split-brain protection is opt-in. There is no designated coordinator node. By default both halves of a partition keep operating; set withMinimumMembers() for a quorum floor that drops a below-quorum minority into a degraded, Down-suppressing mode instead of letting it evict the majority.
×No service registry. refFor() requires a pre-agreed NodeAddress and ActorPath. A receptionist service-registry is planned for C2.
×No automatic rejoin after Down. A node marked Down must restart the process to re-join the cluster.
×Gossip convergence latency. State changes propagate within ~1 gossip interval (default 1 s); rapid membership churn may produce briefly stale views on individual nodes.

For workloads that need consensus — distributed locks, exactly-once semantics, coordinated state — pair the cluster transport with a purpose-built consensus library or delegate to your infrastructure (Redis, etcd, Postgres).

Actors across machines.

The guide walks through two-node wiring, failure detection tuning, TLS setup, and OTel integration end-to-end.

composer require nexus-actors/cluster-tcp

Clustering guide Package reference