Overview

A PicoMQ deployment has three components. Nodes serve clients and hold only caches. An object store holds the data objects. A SQL database holds the metadata log that coordinates the nodes. The write-ahead log lives in either the object store or the SQL database.

Component	Holds	Options
Node	Caches	pico serve process, or a background worker inside Postgres
Object store	Data objects, WAL by default	S3 and compatible stores, local filesystem
SQL database	Metadata log, optionally the WAL	Postgres, SQLite for a single node

Anatomy of a node

Each node runs the same stack. An HTTP listener speaks the Pico protocol or Durable Streams, and a Kafka listener serves Kafka clients. An admin listener serves the admin API and the dashboard. Behind them, an ownership router decides whether this node serves a stream or redirects, the stream service manages the registry of names and per-stream state, and the s3stream engine moves records to and from storage. The same stack runs as a process or as the Postgres extension.

The engine is the only writer of record data. It appends to a write-ahead log for durability, batches records into larger data objects in the background, and serves reads from caches when it can.

The metadata log

All cluster state changes are commands: register a node, create a stream, open it, commit an object, transfer ownership. A node proposes a command to the SQL database, where an ordered log table assigns it a position. Every node tails that table and applies each command to an in-memory state, so all nodes converge on the same view without talking to each other.

The database provides the ordering, which is the property a consensus protocol would otherwise supply. Applying a command is deterministic, so replaying the log always produces the same state. Every 1024 applied commands a node writes a snapshot row and the log below it can be truncated, which keeps replay on startup short. The position of the last applied command is the applied index seen in the admin API.

The applied state answers every routing and placement question: which node owns a stream, which objects exist, which transfers are in flight. Nodes read it from an immutable in-memory view, never from SQL directly, so reads cost nothing and a slow database only delays new commands.

Request routing

Stream names look like URL paths and any node accepts any request. The receiving node checks the view. If the stream is open on another node the client gets a 307 redirect to its advertised address. If it is unowned or local the node serves it directly. Epoch fencing backs this up: a node that lost its registration cannot commit anything, so a stale redirect can waste a hop but never split a stream.

Background work

A few loops run outside the request path. Every node tails the metadata log and watches for stream transfers involving it. One node at a time holds a SQL lease and runs maintenance: expiring abandoned object uploads and deleting destroyed objects from storage. Any node crossing a snapshot interval writes the next snapshot. None of these tasks are special roles, whichever node holds the lease does the work and losing it just moves the work elsewhere.