Skip to content
roxy

Addon model

roxy understands HTTP, not any particular API. Logic that needs to understand the traffic, such as checking a model's tool calls or counting tokens, goes in an addon. Each addon is a step that the request passes through on its way to the rules, and that the response passes back through on its way to the client. An addon is either:

  • a WASM layer (kind: wasm): a WebAssembly component that runs inside roxy (writing addons); or
  • a service layer (kind: service): an external service that each exchange streams through over a WebSocket (service layer protocol).

Configuring addons is covered in addon configuration.

The stack

The stack, top to bottom: client, then CONNECT and TLS, the first addon listed, the second addon listed, the rules, the address floor, the connection to the origin, and the origin. The request runs down through every step and the response runs back up. The two addons are marked as the addons, in the order listed; the address floor and the connection to the origin always run, after the rules. A dotted line between the rules and the address floor marks where capture records what the rules allow, and a flow-log strip down the left is fed by every step.

Addons run in the order they are listed under addons:, between the client and the rules. Nothing configurable runs between the rules and the network, so whatever the rules approved is what leaves. All the steps run at the same time on a stream, each working on the chunk the step before it passed on.

What an addon can do

For each exchange, an addon gets the request as the step before it passed it on. It can:

  • pass the request on unchanged, or change it first;
  • hold it, for example while a person approves it;
  • answer the client itself, without passing the request on;
  • do the same with the response on its way back.

The request and response are handled separately, so an addon can start answering before it has passed the request on, or while it still holds it. An addon passes each request on at most once and answers exactly once. To call anything else, such as a monitoring model, it uses a named endpoint, which does not go through the stack.

An addon can ask for the head only, or the head and the body, in each direction. A body it does not ask for goes past it untouched and costs nothing (what a layer sees).

Guarantees

  1. The rules judge every request that leaves. Whatever the last addon passes on is checked as strictly as a client's request, and judged by the rules as if the client had sent it.
  2. Addons are held to the client's limits. Header limits, body caps and timeouts apply to what an addon passes on, as they do to a client.
  3. Failure is closed. An enforcing addon that fails makes the exchange fail with 503. There is no "on error, pass" (failure).

An addon can replace any response on its way back, including a refusal from the rules. Addons are trusted, the refused request never left, and the flow log keeps the rules' decision.

Ordering across layers

Traffic passes through the addons in order: the second addon sees what the first passed on, in the order it passed it on.

Calls to roxy's host services, such as writing state or adding a tag, run alongside the traffic rather than in it. If addon A writes state that addon B should read, A must wait for its write to finish before passing on the bytes B will read them with. A WASM addon's calls always finish before it carries on. A service layer waits for the result itself.

Two enforce layers, A and B, in roxy's request path, left to right, then the rules. The client's head (a solid line) and body bytes (a dashed line) run through A, then B, then to the rules; between A and B, a chunk reaches B once A sends it. Part-way through the body, A calls state_put, drawn as an arrow down to a host services strip along the bottom with a result arrow back up, and the note awaits result, then sends on; A's body line is broken at that point. In B, on the chunk that followed, a state_get arrow goes down to the strip and a result comes back, with the note on that chunk: sees A's write. A dotted line between the two calls says ordered only by A's wait. The strip lists host services: add_tag, state_put, state_get, record, and more.

For the rules to see a tag or state write when they decide on the request, it must land before the addon passes the request on. A later write is seen by the next rule that reads it, but never changes a decision already made.

Modes

  • mode: enforce (the default): the addon is in the path, and what it does takes effect.
  • mode: observe: the addon gets a copy of the traffic and cannot change or delay it. Its failures are logged, never fatal. This is the way to try out a new monitor. An observer that falls too far behind loses its copy, and the exchange carries on (observers).
Edit on GitHub