Run the gateway yourself

By default aigw serves your traffic from gateways we operate. On the Enterprise plan you can run the gateway inside your own network instead. Requests are then served only by your gateways, and prompt and response content never crosses your boundary.

This is aimed at organisations that cannot send request content to a third party: regulated industries, critical infrastructure, and anyone whose data-residency rules are stricter than “the provider is in the right region.”

What crosses your boundary, and what does not

The gateway keeps one outbound HTTPS connection to the aigw control plane’s API, the same address the console talks to. Nothing connects inward, no other port is used, and a corporate proxy that passes HTTPS passes this.

Comes down: your configuration. Which models are available, which virtual keys exist and what they may do, budgets, and your governance policy.

Goes up: telemetry, as metadata only. Model, token counts, cost, latency, status, and governance flags such as “this request was blocked.” That is what fills your dashboards, budgets and audit trail.

Never leaves your network: the request and the response. Prompt text, completions, tool arguments, tool results. Content logging is opt-in and, when you run your own gateway, either stays off or stays local.

If you turn on your own gateways, the gateways we operate stop receiving your workspace entirely. Your configuration is not distributed to them, and they cannot serve your traffic even if a request reached them.

What you need

• Somewhere to run a container. One machine with Docker is enough for an evaluation; a Kubernetes cluster with two small replicas is the usual production shape. The gateway is stateless and keeps only a cache of its configuration on disk.
• Outbound HTTPS from there to the aigw control plane. Nothing needs to be exposed inbound, and we do not need to know your address.
• Network access from there to whichever model providers you use, or to your own self-hosted models.

The gateway is a single static container with no shell and no package manager, running as a non-root user on a read-only filesystem. It is built for amd64 and arm64, so Graviton, Ampere and similar nodes are fine.

Rolling one out

Everything below is printed for you, filled in with your own values, under Settings → Gateways in the console. The commands here are the same ones, so you can read the shape of it before you start.

The order matters, and the console holds you to it: a gateway has to be connected before you can move your traffic to it, so there is no way to cut yourself off.

1. Get credentials for the image

The chart and the image are not public. Under Settings → Gateways, choose Get registry credentials. You get one credential for your whole workspace, used by Helm when you install and by Kubernetes every time a pod starts.

helm registry login registry.aigw.app -u aigw -p <registry-token>

kubectl create namespace aigw
kubectl create secret docker-registry aigw-registry -n aigw \
  --docker-server=registry.aigw.app \
  --docker-username=aigw \
  --docker-password=<registry-token>

On a single machine with Docker, one login is enough:

docker login registry.aigw.app -u aigw -p <registry-token>

The credential does not expire, and asking for it again returns the same value, so there is nothing to lose or rotate.

2. Add a gateway

Still under Settings → Gateways, add a gateway. You get two values, shown once: the token it authenticates with, and the key its provider credentials are sealed to. Neither can be recovered afterwards, so if you lose them, add another gateway and revoke the old one.

3. Install it

In a cluster:

kubectl -n aigw create secret generic aigw-edge \
  --from-literal=edge-token=<edge-token> \
  --from-literal=edge-key=<edge-key>

helm install aigw oci://registry.aigw.app/aigw/charts/aigw-gateway -n aigw \
  --set controlPlane.api=https://api.aigw.app \
  --set controlPlane.tenant=<your workspace id> \
  --set secrets.existingSecret=aigw-edge \
  --set image.repository=registry.aigw.app/aigw/gateway \
  --set imagePullSecrets[0].name=aigw-registry

On one machine, put the values in a .env file next to the compose file the console prints, and start it:

AIGW_CONTROL_PLANE_API=https://api.aigw.app
AIGW_EDGE_TOKEN=<edge-token>
AIGW_EDGE_KEY=<edge-key>
AIGW_TENANT=<your workspace id>

docker compose up -d

Nothing else has to be reachable. The gateway opens one outbound HTTPS connection and appears in the console’s list within a few seconds of starting.

4. Move your traffic

Your traffic is still served by us at this point. Once your gateway shows as connected, choose Use my own gateways. From then on your configuration is distributed only to your gateways and ours stop receiving your workspace at all.

Switching back is the same step in reverse.

Pointing your applications at it

Your applications point at the gateway as their OpenAI-compatible base URL, with an aigw virtual key as the API key. In a cluster that is the Service:

http://aigw-gateway.aigw.svc.cluster.local:8080

Everything else works exactly as it does on the hosted gateways. Same API, same virtual keys, same budgets, same DLP, same audit trail.

Behind a proxy

The gateway honours HTTPS_PROXY and NO_PROXY like any other process. If your proxy inspects TLS and presents your organisation’s certificate for outside hosts, either exempt api.aigw.app from inspection, or give the gateway your issuing CA: proxy.caSecret in the chart, or proxy-ca.pem next to the compose file. Without it the gateway refuses the proxy’s certificate rather than talk to something it cannot verify. Everything the gateway sends is metadata and sealed credentials, so exempting the host gives an inspecting proxy nothing it could have read anyway.

Client addresses behind a load balancer

Source-address rules and the audit trail use the caller’s address. The gateway takes it from X-Forwarded-For only when the connection comes from a trusted proxy, and then reads the nearest address that is not one, so a client cannot name itself by writing the header. By default loopback and the private ranges are trusted, which fits a load balancer inside your own network. If clients on that network must not be able to claim another address, set AIGW_TRUSTED_PROXIES to your load balancers’ addresses or ranges, comma separated.

Scaling and failure

The gateway holds its configuration in memory and replays it from the control plane on start, so replicas need no coordination and scale horizontally. The chart ships a PodDisruptionBudget and spreads replicas across nodes by default.

A gateway that has never reached the control plane will not serve: it waits rather than answering requests with empty policy. One that has serves from its local copy straight away on restart, and a control-plane interruption does not stop it serving; it keeps using the configuration it last confirmed and picks up changes when the connection returns.