Skip to content
roxy

Quickstart

This runs clients through roxy with three controls on every model call: an addon that authenticates the client, an addon that holds each user to a token quota, and an inspect_sentinel sidecar that blocks tool calls matching a denylist. roxy gets its policy and the model's API key from a small control plane (node mode). The model is scripted and the traffic generated, so nothing needs an API key. The setup is in examples/quickstart.

Prerequisites

  • Docker with Docker Compose
  • Git

Clone the repo

git clone https://github.com/roxy-proxy/roxy-proxy.git
cd roxy-proxy/examples/quickstart

Start the stack

docker compose up --build --attach traffic

The first run builds the images, which takes a few minutes. Ctrl-C stops the stack. It runs:

  • controlplane: gives roxy its config, roxy.yaml, and the model's API key, and prints the flow log roxy sends back.
  • roxy: the proxy, on 127.0.0.1:3128.
  • fake-model: a scripted model API. It refuses requests without the API key, which only roxy has.
  • auth-service: knows alice-secret as alice and bob-secret as bob.
  • quota-board: the quota service. Each user has 600 tokens, refilling at five a second.
  • sentinel: the inspect_sentinel sidecar.
  • inspect-view: Inspect's log viewer.
  • traffic: the clients. Alice calls every six seconds, bob every twenty, and mallory, whom the auth service does not know, every thirty.

Watch the traffic

Each client uses the Anthropic SDK through roxy, with its credential in an x-roxy-auth header. The scripted responses include a Bash tool call that runs curl, which the sentinel blocks:

traffic-1  | [alice #2] I will fetch that page for you. roxy's sentinel blocked a Bash tool call (denylist match: 'curl').

Responses stream to the client as the model produces them, except tool calls. The sentinel holds each tool call until its arguments are complete, then lets it through or replaces it with a note. Mallory is refused by the auth check:

traffic-1  | [mallory #1] 401 AuthenticationError: credential refused by the auth service

Within the first minute alice uses up her tokens, and her calls are refused until they refill:

traffic-1  | [alice #8] 429 RateLimitError: token bucket empty (600 tokens, refills 5/s); try again in 15s

Watch the quota board

Open http://127.0.0.1:8090 to see each user's tokens. To make a call of your own:

docker compose run --rm client --user bob
docker compose run --rm client --user mallory --secret wrong

Watch it in Inspect View

Open http://127.0.0.1:7575. Each client is a sample. Its transcript shows every model call, the tool calls, and the sentinel's verdict on each. Reload the page to see new calls.

Inspect View: a curl tool call, and the sentinel's rejection under it

Read roxy's flow log

docker compose logs roxy

Each request has one line: the rule it matched, the addons it went through, the user:alice tag the auth check set, and the x-api-key header the rule added. The addons' calls to their services are logged too. The control plane receives the same events every few seconds; docker compose logs controlplane shows them.

Stop the control plane

roxy's policy is valid for sixty seconds and refreshed every fifteen. Stop the control plane:

docker compose stop controlplane

Within a minute the policy expires and every call fails with 403. docker compose start controlplane brings a fresh policy and roxy recovers without a restart.

Clean up

Ctrl-C in the terminal running up, then:

docker compose down -v

What just happened

An exchange passes through the addons in the order roxy.yaml lists them, then the rules.

A model call from the client passes through the auth gate, the token quota and the sentinel, then the rules and the connector, to the fake model; each layer calls its own service, and the flow log records the exchange
  • The auth check (addons/auth-gate) asks the auth service who the x-roxy-auth credential belongs to, tags the exchange user:alice, removes the header and passes the request on. An unknown credential is answered 401.
  • The token quota (addons/token-quota) asks the quota board whether the user has tokens left and answers 429 if not. As the response streams it reads the token usage and reports it to the quota board. Usage is only known after the response, so a user can go over by one call.
  • The sentinel is a service layer: an addon that runs in its own process. It checks every tool call in a model response with inspect_sentinel.

The rules then allow exactly one request, POST /v1/messages on the model host, and add the real API key from a secret. A request the addons skip is one the rules deny.

If an addon cannot do its job, for example because its service is down, the exchange fails with 503. Stop the quota board (docker compose stop quota-board) to see it.

To run roxy from a config file instead of the control plane:

docker compose -f compose.yaml -f compose.standalone.yaml up --build --attach traffic

The denylist

The sentinel blocks tool calls matching a regular expression from policies.py. The default catches curl, wget, nc, ncat, ssh, scp, rsync and rm -rf. Set your own with SENTINEL_DENY:

SENTINEL_DENY='\bls\b' docker compose up --attach traffic

Using it with a real client

Any HTTP client that honours HTTP_PROXY can take the clients' place. For Claude Code, against the scripted model:

HTTP_PROXY=http://127.0.0.1:3128 \
ANTHROPIC_BASE_URL=http://fake-model:8080 \
ANTHROPIC_API_KEY=placeholder \
ANTHROPIC_CUSTOM_HEADERS='x-roxy-auth: alice-secret' \
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 \
claude

A client follows HTTP_PROXY because it chooses to. To contain an agent, run it where its only route out is roxy (sandbox containment).

Next steps

Edit on GitHub