AI Gateway

The Netzilo AI Gateway sits between your agents and the LLM providers and MCP servers your organization allows. Agents and tools such as Claude Code, Cursor, Codex, an OpenAI or Anthropic SDK, or a CI script call the gateway instead of the provider, with the user's Netzilo token as the API key. The gateway identifies every caller as a Netzilo user, applies that user's gateway filters, and forwards the request with the provider key it holds. Provider keys never leave the gateway host.

The gateway is the second place Netzilo governs AI traffic. The first is the Netzilo client on a device, see AI Detection & Response. Use both on developer workstations, and the gateway alone for servers, CI jobs and devices you do not manage.

You run the gateway yourself, on Netzilo Cloud and on a self-hosted Netzilo Server. It is an ordinary peer of your account. You configure it only in the dashboard: Edge > Providers, Edge > Providers, Edge > Governance > Guardrails, Edge > Governance > Limits and Edge > Governance > Filters. Users find it in the Workplace's AI Workspace, where they can also chat with the granted models and tools.

Watch it work

AI Gateway · 2:34

Govern model access, budgets and spend

Keep provider keys in the gateway, grant a team only its approved models and cap its daily spend.

Read: AI Gateway →YouTube ↗

AI Gateway · 1:43

Reroute instead of refuse

Serve requests for an unapproved model with an approved one, so developers keep working within budget.

Read: Detection Rules: actions →YouTube ↗

How it works

Diagram of the AI Gateway: agents send a request with a bearer token through the tunnel door, a published application or the public door; the gateway identifies the user, checks the model grant, the limits and the guardrails, and forwards the request to the LLM providers or MCP servers with the key it holds; the Netzilo Server sends the gateway its policy and receives usage facts

For every request the gateway does the following, in this order:

  1. Identify the user. The caller sends Authorization: Bearer <token>. On the tunnel door the calling peer also identifies the user. A caller the gateway cannot identify on the public door is refused with HTTP 401.
  2. Check the model. One of the user's gateway filters must grant the model. A bare model name goes to the granted provider that serves it; provider/model must be granted for that provider. Without a grant the request is refused with HTTP 403.
  3. Apply limits. Spend, token and request quotas from the user's filters. When several filters apply, the lowest value of each kind wins. An exhausted limit answers HTTP 429 or 402.
  4. Run guardrails. The guardrails bound by the user's filters run on the prompt, on the answer, on each tool call the model asks for, and on each MCP tool call and its result. A guardrail can block, redact, or send the prompt to another model.
  5. Record. One usage record per model request and per MCP tool call: who, which model, tokens, cost, latency and status. Separately, the conversation after the guardrails, for the session's trajectory (see What the gateway sends).

The gateway keeps each user's filters for 60 seconds and re-checks everyone when a policy changes. Inside a request, an internal error lets the request through; the identity check at the door is never relaxed.

API surfaces

Point each client at the path it expects. <gateway> is the address users reach, for example https://aigw.apps.example.com.

PathUsed byModel grant and guardrailsLimitsUsage recorded
<gateway>/v1/chat/completionsOpenAI SDKs, Cursor, LangChain, curlyesyesyes
<gateway>/v1/responsesOpenAI Responses API, Codexyesyesyes
<gateway>/anthropic/v1/messagesClaude Code, Claude Desktop, Anthropic SDKyesyesyes
<gateway>/v1/embeddings, <gateway>/v1/completionsembedding and legacy completion callsno, recorded onlyyesyes
<gateway>/mcp/<slug>MCP clientsyes, on tool calls and resultsyesyes
<gateway>/v1/modelsany clientlists the models the user may use-no

Streamed answers are released about 1 KB behind the model, so that a redaction applies before the text is shown. Users see a short delay on long answers. A block ends the stream.

Before you start

  • A setup key for the gateway's own peer, see Setup Keys. Its auto-group decides which group the gateway peer joins, for example AI-gateways.
  • A service token. Create a service user under Team > Agents with the role Admin, then create an access token for it, see Agents. The service token binds callers to your account and loads filters, providers and MCP servers. Without it the gateway refuses every chat request.
  • At least one provider under Edge > Providers, see Providers.
  • A gateway filter under Edge > Governance > Filters with Filter Type set to Gateway: the users' groups, at least one model on the Models tab, and the gateway's address on the Gateways tab, see Filters.
  • Outbound access from the host: TCP 443 to the Netzilo Server, and HTTPS to the providers and MCP servers you define.

Step 1: Install the gateway

The Add Peer dialog under Endpoint > Peers shows the same commands on the Gateway tab of the Linux, macOS and Windows tabs.

The Install Netzilo Client dialog on the Linux tab with the Gateway sub-tab selected: step 1 is the install_aig.sh command, step 2 the netzilo-aigw up command with the management URL and placeholders for the setup key and the service token

Linux and macOS

Install the gateway, then start it as a service with the setup key and the service token:

curl -fsSL https://pkg.netzilo.com/download/linux/install_aig.sh | sh

sudo netzilo-aigw up \
  --management-url https://srv.netzilo.com \
  --setup-key <SETUP-KEY> \
  --service-pat <SERVICE-TOKEN> \
  --ai-listen :8447

On macOS use the same script from https://pkg.netzilo.com/download/macos/install_aig.sh. The script installs netzilo-aigw into /usr/local/bin. up installs the gateway as a service (systemd on Linux, launchd on macOS), starts it and waits until it is connected:

Netzilo AI Gateway started as the netzilo-aigw service; settings in /var/lib/netzilo-gateway/netzilo-aigw.json
Waiting for it to connect ... connected.
  mode:     node
  peer:     aigw1
  sessions: 0

The gateway keeps running after you log out, restarts if it stops, and starts again at boot. up takes every flag of netzilo-aigw run and saves them, readable by root only. Flags you give again replace the saved ones, so sudo netzilo-aigw up --service-pat <NEW-TOKEN> rotates the token, and a bare sudo netzilo-aigw up restarts the gateway with what is saved.

To do both in one step, pass the flags to the script:

curl -fsSL https://pkg.netzilo.com/download/linux/install_aig.sh | sh -s -- \
  --management-url https://srv.netzilo.com \
  --setup-key <SETUP-KEY> --service-pat <SERVICE-TOKEN> --ai-listen :8447

Run the script again to upgrade: a gateway that is already set up restarts on the new version with its saved settings.

CommandWhat it does
sudo netzilo-aigw up [flags]Saves the flags, installs or updates the service, starts it and waits until it is connected
netzilo-aigw statusWhether the service runs and is connected
sudo netzilo-aigw downStops the gateway; it starts again at the next boot
sudo netzilo-aigw uninstallStops and removes the service; its settings and identity stay, so a later up brings back the same gateway

The log is in the journal on Linux (sudo journalctl -u netzilo-aigw) and in /Library/Application Support/Netzilo/gateway/netzilo-aigw.log on macOS.

A gateway set up with an earlier version of the script, through /etc/netzilo-aigw/aigw.env, is moved to the new service by the next up, or by running the script again. Its flags and secrets are taken over and it stays the same gateway.

To try the gateway in the foreground instead, run sudo netzilo-aigw run with the same flags; it stops when the terminal closes.

Windows

In an administrator PowerShell:

irm https://pkg.netzilo.com/download/windows/install_aig.ps1 | iex

netzilo-aigw up `
  --management-url https://srv.netzilo.com `
  --setup-key <SETUP-KEY> `
  --service-pat <SERVICE-TOKEN> `
  --ai-listen :8447

The script installs netzilo-aigw.exe into %ProgramFiles%\Netzilo AI Gateway and adds the folder to the PATH. up installs the netzilo-aigw Windows service, starts it and waits until it is connected; status, down and uninstall work as on Linux. The log is %ProgramData%\Netzilo\gateway\netzilo-aigw.log. For one step, set the flags first:

$env:AIG_UP_ARGS = "--management-url https://srv.netzilo.com --setup-key <SETUP-KEY> --service-pat <SERVICE-TOKEN> --ai-listen :8447"
irm https://pkg.netzilo.com/download/windows/install_aig.ps1 | iex

Container

docker run -d --name netzilo-aigw --restart unless-stopped \
  -p 8447:8447 -v netzilo-aigw:/var/lib/netzilo-gateway \
  -e NZ_GATEWAY_SETUP_KEY=<SETUP-KEY> -e NZ_GATEWAY_SERVICE_PAT=<SERVICE-TOKEN> \
  ghcr.io/netzilo/net-aigateway:latest run \
  --management-url https://srv.netzilo.com --workplace-url https://go.netzilo.com \
  --name aigw1 --listen 127.0.0.1:8443 --ai-listen :8447 --ai-insecure-plain-http

The image is linux/amd64, runs as a non-root user and needs no TUN device and no extra capabilities. Keep the volume: it holds the peer key.

Self-hosted Netzilo Server

Replace both addresses with your own domain: --management-url https://<your domain> and --workplace-url https://<your domain>. See Install on your own server.

Flag reference

Secrets can come from the environment instead of flags: NZ_GATEWAY_SETUP_KEY, NZ_GATEWAY_PAT and NZ_GATEWAY_SERVICE_PAT.

FlagDefaultWhat it does
--management-urlrequiredhttps://srv.netzilo.com on Netzilo Cloud, https://<your domain> self-hosted. Scheme and host only
--workplace-urlthe management URLThe Workplace users sign in at. Set https://go.netzilo.com on Netzilo Cloud
--setup-key or --patCredential for the gateway's own peer. Give one, never both
--service-patService token that binds callers to your account and loads policy
--namehost nameThe gateway peer's name in Endpoint > Peers and in usage reports
--data-dir/var/lib/netzilo-gateway (Linux)Peer key, applied state and the usage outbox
--ai-tunnel-port8447Tunnel door on the peer's tunnel address. 0 turns it off
--ai-listenoffPublic door, for example :8447
--tls-cert, --tls-keyCertificate and key for HTTPS on the listeners. Give both
--tls-cert-dir, --tls-domainTake the certificate for a domain from a Caddy storage directory
--ai-trusted-proxiesCIDRs of a TLS-terminating proxy in front of --ai-listen
--ai-insecure-plain-httpoffSilence the warning about a plain-HTTP public door
--ai-allow-unattributedoffOn the public door, pass requests from unknown callers through ungoverned instead of 401
--ai-policy-refresh60sHow long a user's filters are trusted before the gateway asks again
--listen:8443The browser-access proxy used by clientless access. It is always on: bind it to 127.0.0.1:8443 when you only need the AI Gateway
--admin-listen127.0.0.1:9090Health endpoints /healthz and /readyz
--log-levelinfoLog verbosity

Step 2: Make the gateway reachable

Choose one or more ways in. Then put the address users should use on the Gateways tab of the gateway filter, so that the Workplace shows it.

The Edit Filter dialog of a Gateway filter on the Gateways tab, with tabs Targets, Models, Limits, MCP, Gateways, Guardrails and Name, and one gateway address https://llm.apps.example.com

Inside the network (tunnel door)

Devices with the Netzilo client reach the gateway at http://<gateway-name>.netzilo.network:8447, or at the gateway peer's 100.x address. No certificate is needed: the traffic is inside the encrypted tunnel. The caller is identified by its peer, so a request without a token is still attributed to the device's user.

The users' devices need a network policy to the gateway peer on TCP port 8447. The Default policy (All to All) covers it. In a hardened account, add a policy under Network > Policies from the users' groups to the gateway's group on TCP 8447, see Policies. Keep the gateway reachable only from groups that should use it: a request from a peer the gateway cannot map to a user passes ungoverned.

From anywhere (published application)

This is the recommended public exposure. No public door and no certificate are needed on the gateway host.

  1. Go to Network > Applications and click Add Application, see Applications.

  2. Fill in the dialog:

    FieldWhat to enter
    Addressaigw under one of your application domains, for example aigw.apps.example.com
    Targethttp://<gateway-name>.netzilo.network:8447. The port is required: without it the proxy dials port 80 and answers 502
    GroupsThe groups of the users who call the gateway
    Forward the user's credentialOn
    Send the address as Host headerOff
  3. Users call https://aigw.apps.example.com/v1/chat/completions with Authorization: Bearer <token>. The reverse proxy admits the token and forwards the request to the gateway through the user's own tunnel.

The published address also serves the sign-in used by MCP connectors and by Claude Desktop, so both work without a public door.

Public door

Start the gateway with --ai-listen :8447 on a host with a public address and open port 8447 in the firewall. Every caller must send a bearer token; the public door never identifies callers by address. Give the door a certificate with --tls-cert and --tls-key, or put a TLS proxy in front of it and list the proxy in --ai-trusted-proxies. Without TLS the user's token travels in clear text.

Step 3: Connect agents and tools

Users find their gateways, models, MCP servers and connection snippets in the Workplace, in the Gateways & MCP Servers panel of the AI Workspace, and create their token there with Create Access Token. See Agentic Access. They can also chat with the granted models and MCP tools right there, see AI Workspace.

The Gateways & MCP Servers panel of the AI Workspace with one LLM gateway, Acme AI Gateway 1 at https://ai.example.com, and four MCP servers below it

Every client sends the token as a bearer token:

HeaderAccepted values
Authorization: Bearer <token>A user's personal access token (nzl_…), the Workplace's AI Gateway token, or a token the client got by signing in at the gateway

Use the token as a bearer token. With the Anthropic SDK pass it as auth_token, not api_key. A token of another Netzilo account is refused.

ClientBase address
OpenAI-compatible clients (Python, Node.js, Cursor, LangChain)<gateway>/v1
Claude Code (ANTHROPIC_BASE_URL), Anthropic SDK<gateway>/anthropic
curl<gateway>/v1/chat/completions
MCP clients<gateway>/mcp/<slug>

Check that a user's token works:

curl https://aigw.apps.example.com/v1/models -H "Authorization: Bearer <token>"

The answer lists the models the user's filters grant. Each model is named <id>/<model>: the provider's ID under Edge > Providers, a slash and the model's name, for example anthropic-production/claude-sonnet-4-6. Clients send the model by that name on every gateway. See Model IDs.

Claude Desktop: Sign in with your organization

Claude Desktop can sign in at the gateway with the user's Netzilo account, without a token to paste.

  1. In Claude Desktop, set the gateway endpoint to exactly <gateway>/anthropic, for example https://aigw.apps.example.com/anthropic.
  2. Choose Sign in with your organization.
  3. Claude shows a code of the form XXXX-XXXX and opens the approval page in the browser.
  4. Sign in to the Workplace if asked. Check that the code on the page matches the code in Claude, then click Allow. The page says that access is allowed; nothing redirects.
  5. Claude collects its token within a few seconds.

The token Claude receives is the user's own access token named Claude (MCP), valid for one day and renewed automatically for up to 30 days. Users see it under Profile Settings and can revoke it there; Claude then has to sign in again.

Only <gateway>/anthropic serves this sign-in. With <gateway>/v1 or <gateway>/openai Claude reports gateway SSO discovery failed: 401. With the bare <gateway> address the sign-in works but requests fail with HTTP 405.

What users see when a request is refused

Refusals, in the order a caller can meet them:

HTTP statusMessageCause and fix
401a Netzilo credential is requiredNo token, a malformed or expired token, or a token of another account, on the public door or a published address
503management is not reachable, try againThe gateway could not ask the Netzilo Server who the token belongs to. Retry
404not served by the AI gatewayA path the gateway never forwards
405Method Not Allowed on POST /v1/messagesThe Anthropic API is served under /anthropic: set the client's base address to <gateway>/anthropic
502the AI gateway upstream is not reachableThe gateway's built-in upstream is down. Read the gateway log
400no model is granted to this user or model <m> is not granted to this userNo gateway filter grants that model to the user. Grant it under Edge > Governance > Filters
400the guardrail's reason, default blocked by Netzilo policyA guardrail blocked the prompt, the answer, a tool call or an MCP call. The event in Activity names the rule
400Netzilo Guardrail: <file name>: <reason>A guardrail refused a file the request carries, see Guardrails. The next message goes through without the file
503Netzilo AI Gateway: too many large requests at once; retry shortlyToo many large requests (files) at once; the client retries after 5 seconds. See Large requests and files
429 or 402budget_exceeded, rate_limited, token_limited or request_limitedA limit from the user's filters is used up for its window, see Limits
MCP tool answerConnect <server> in your Workplace before using its toolsThe MCP server needs the user's own sign-in. The user connects it in the Workplace's AI Workspace, see Agentic Access

Policy refusals have the type netzilo_policy. Gateways before October 2026 answered 403 instead of 400. A fallback provider is never used to get around a refusal.

What the gateway sends to the Netzilo Server

  • Usage facts, every 15 seconds: per model request the user, gateway, provider, model asked for and model served, tokens, cost in US dollars, latency and status; per MCP tool call the server, tool, status, latency and the sizes of arguments and results. Usage facts also name the MCP client and its version, as it introduced itself, and the MCP session. Usage facts carry no prompt, answer, argument or result text. They survive an outage or a restart on disk and are never counted twice.
  • Captured conversations, sent apart from the usage facts: for each model call, the messages it added to its session (system prompt, prompts, the agent's context, thinking, answers, tool calls and their results) as the caller sent and received them, after the guardrails, and for each MCP tool call its arguments and its result, after the guardrails. What a guardrail redacted arrives redacted, and a tool call the guardrails held back or refused is not sent. A message longer than 32 KB is cut. A message a session sent before is sent once, not with every call. Captured conversations are held in memory only, never on disk: those still waiting when the gateway stops, or beyond 64 MB during an outage, are lost.
  • Guardrail events, like a device does: when a rule matches, the event carries the rule name and the content that matched.
  • Files attached to model requests are read in the gateway's memory and never sent to the Netzilo Server; captured conversations hold only the text of messages. The session records each file's name, type, size, SHA-256 and verdict, and a guardrail event about a file carries the file's text as the guardrails left it, up to 8 KB.

Usage appears at once in the AI Usage report and in Dashboard > Agentic Activity when its numbers are next built (hourly, or at once with Refresh). Captured conversations appear in the session's Trajectory and MCP Calls, under Edge > Sessions and in the Agentic Activity window, which only administrators open, see Dashboard. A device whose Netzilo client also inspects this traffic does not record it again: a call that reaches the gateway is recorded once, by the gateway. Guardrail verdicts appear in Activity > Events under the category AI Edge, see Events.

Operate the gateway

The gateway peer. It appears under Endpoint > Peers with its --name, version and connection state, in the groups of its setup key. If the peer is deleted, the process exits so that the service manager restarts it, and it registers as a new peer: add its groups again.

Health. curl http://127.0.0.1:9090/healthz answers ok. /readyz answers ready, or 503 with the reason, such as management unreachable.

Logs. On Linux: sudo journalctl -u netzilo-aigw -f. Lines worth knowing:

Log lineMeaning
gateway: bound to an account with N usersThe service token works
gateway ai: listening on tunnel <ip>:8447 …The tunnel door is open
gateway ai: policy …: N user(s) cached, N filter(s) held, N rule(s) loaded, N failedFilters and guardrails loaded. failed counts guardrails whose YAML did not parse
gateway ai: refused <model> for <user>: <reason>A model or limit refusal
gateway ai: no --service-pat: …The service token is missing: chat requests are refused
gateway: no data dir: …The data directory is not writable: every restart registers a new peer

Upgrade. Run the install script again, or pull a newer image tag. The script restarts a gateway that is already set up on the new version, with its saved settings. The data directory keeps the peer key, so the gateway returns as the same peer. netzilo-aigw --version prints the installed version.

Large requests and files

The gateway admits large requests and reads attached files within two memory budgets, sized from the memory of its container or host:

VariableDefaultWhen it is full
NZ_AI_BODY_BUDGET_MBmemory ÷ 48, between 32 and 512 MBthe request waits up to 10 seconds, then gets 503 too many large requests at once with Retry-After: 5
NZ_AI_FILE_BUDGET_MBmemory ÷ 256, between 16 and 128 MBthe file waits up to 2 seconds, then is sent unread, judged on its type facts only (unscannable_reason: not_read)

File analysis fails open: a guardrail on unscannable_reason: not_read reports such files, or blocks them where an unread file must never pass. Set the variables in the container (-e NZ_AI_FILE_BUDGET_MB=64), or for the Linux service with sudo systemctl edit netzilo-aigw ([Service] and Environment=NZ_AI_FILE_BUDGET_MB=64) and a restart.

Rotate the service token. Create a new token for the service user, run sudo netzilo-aigw up --service-pat <NEW-TOKEN>, then revoke the old token. Users' spend counters survive the restart.

Troubleshooting

A file the user sent was not blocked or redacted

Open the session under Edge > Sessions: the model call lists its Files. No file there means the client sent text, not a file (Claude Code reading a CSV, Claude Cowork), and text guardrails apply. A file marked not scanned gives the reason, such as an encrypted or scanned PDF. Otherwise check that a filter for the user's groups binds a rule for files.

The service exits at once with a service PAT is required or give either a setup key or a PAT

Give the service token, and exactly one of the setup key or the PAT for the gateway peer: sudo netzilo-aigw up --service-pat <SERVICE-TOKEN> --setup-key <SETUP-KEY>.

The Workplace shows the gateway offline although curl works

The browser's check is refused for its origin: --workplace-url is missing or wrong. On Netzilo Cloud set --workplace-url https://go.netzilo.com and restart. The card recovers at its next check, within 30 seconds.

A valid token gets 401 a Netzilo credential is required

The token belongs to another account, has expired, or was pasted with a space. Create a new token and try GET /v1/models again.

Every request gets 403 no model is granted to this user

The user is in no group that a gateway filter targets, the filter grants no model, the filter is disabled, or its type is Endpoint. Fix the filter under Edge > Governance > Filters.

A published address answers 502 The application did not answer.

The target lacks :8447, the gateway is down, or the users' device groups have no policy to the gateway peer. Fix the target or the policy.

A published address answers 404 There is no application at this address.

The application is disabled or not published yet. The proxy refreshes its list within 30 seconds.

Claude Desktop: gateway SSO discovery failed: 401 or Gateway returned HTTP 405

The endpoint in Claude is not exactly <gateway>/anthropic. Correct it and sign in again.

Claude Desktop: token_endpoint: must be same-origin or server does not advertise device_authorization_endpoint

The gateway, or the reverse proxy in front of a published address, is older than the organization sign-in. Upgrade it.

The approval page says "That code was not found or has expired."

The code was mistyped, or more than 10 minutes passed since Claude showed it. Retype it, or start the sign-in again in Claude.

Claude keeps waiting after Allow

Claude checks every 5 seconds and must collect its token within 5 minutes of Allow. After a Netzilo Server restart, or with several management replicas without session affinity, the pending sign-in is lost: sign in again.

Usage never appears in the dashboard

Look for gateway ai: usage: … kept, upload failed in the log: the gateway cannot reach the Netzilo Server or its service token is refused. The service token must belong to a service user with the role Admin: the Netzilo Server refuses usage and conversations from any other token. The dashboard panel is built hourly, or at once with Refresh; the AI Usage report reads the same data at once.

Every restart creates a new peer

The log shows gateway: no data dir. Make --data-dir writable for the service user.

Limits to know

  • Files are judged as files only here, and only when the client sends them as files. Text a client extracts itself (Claude Code reading a CSV, any document in Claude Cowork) is judged as text. A PDF or a spreadsheet value cannot be redacted in place, so a redaction there refuses the request. There is no OCR. A file over 25 MB or with over 5 MB of text is judged on its type facts.
  • A limit applies per user per gateway. Two gateways in one account count separately, and there is no pool shared by a group. Changing a user's grants or limits restarts their counters.
  • Embeddings and legacy completions are identified, limited and recorded, but guardrails do not run on them.
  • A revoked token works for up to one minute at the gateway, which caches identities for that long.
  • Sign-ins at the gateway live in one management process for 10 minutes. A self-hosted server with several management replicas needs session affinity.
  • Windows has no service for the gateway, and there is no Linux arm64 build.
  • The browser-access proxy on port 8443 is always on. Bind it to 127.0.0.1:8443 when you do not use clientless access.
  • AI Detection & Response - Devices and the gateway, one rule language
  • Providers - The models the gateway routes to
  • MCP - MCP servers and bundles the gateway serves
  • Limits - Spend, token and request quotas
  • Filters - Gateway filters grant models, limits and MCP servers
  • Guardrails - The rules the gateway applies
  • Agentic Access - What users see and paste
  • Applications - Publish the gateway at an HTTPS address
  • AI Usage - Requests, tokens and cost per user and model