Run the gateway yourself

By default aigw serves your traffic from gateways we operate. On the Enterprise plan you can run the gateway inside your own network instead. Requests are then served only by your gateways, and prompt and response content never crosses your boundary.

This is aimed at organisations that cannot send request content to a third party: regulated industries, critical infrastructure, and anyone whose data-residency rules are stricter than “the provider is in the right region.”

What crosses your boundary, and what does not

The gateway keeps one outbound connection to the aigw control plane. Nothing connects inward.

Comes down: your configuration. Which models are available, which virtual keys exist and what they may do, budgets, and your governance policy.

Goes up: telemetry, as metadata only. Model, token counts, cost, latency, status, and governance flags such as “this request was blocked.” That is what fills your dashboards, budgets and audit trail.

Never leaves your network: the request and the response. Prompt text, completions, tool arguments, tool results. Content logging is opt-in and, when you run your own gateway, either stays off or stays local.

If you turn on your own gateways, the gateways we operate stop receiving your workspace entirely. Your configuration is not distributed to them, and they cannot serve your traffic even if a request reached them.

What you need

• A Kubernetes cluster. Two small replicas are enough to start; the gateway is stateless and holds nothing on disk.
• Outbound TLS from that cluster to the aigw control plane. Nothing needs to be exposed inbound. We confirm the endpoint and permit your egress during onboarding, so have the cluster’s outbound address to hand.
• Network access from the cluster to whichever model providers you use, or to your own self-hosted models.

The gateway is a single static container with no shell and no package manager, running as a non-root user on a read-only filesystem. It is built for amd64 and arm64, so Graviton, Ampere and similar nodes are fine.

Getting the chart

The Helm chart and the gateway image are not public. Running your own gateway is an Enterprise capability, and we hand over the chart along with credentials for the image registry when it is enabled for your workspace. Talk to us and we will set it up with you.

Installing

Add a gateway under Settings → Gateways in the console. You get a one-time set of values to pass to the chart, and the gateway appears in that list as soon as it connects.

helm install aigw <chart> -n aigw --create-namespace \
  --set controlPlane.natsURL=<given during onboarding> \
  --set secrets.existingSecret=aigw-edge-credentials

Your applications then point at the in-cluster Service as their OpenAI-compatible base URL, with an aigw virtual key as the API key:

http://aigw-aigw-gateway.aigw.svc.cluster.local:8080

Everything else works exactly as it does on the hosted gateways. Same API, same virtual keys, same budgets, same DLP, same audit trail.

Scaling and failure

The gateway holds its configuration in memory and replays it from the control plane on start, so replicas need no coordination and scale horizontally. The chart ships a PodDisruptionBudget and spreads replicas across nodes by default.

A gateway that cannot reach the control plane on startup will not serve: it stops rather than answering requests with stale or empty policy. Once running, a control-plane interruption does not stop it serving; it keeps using the configuration it last confirmed.

Switching over

You can add gateways and watch them connect while your traffic is still served by us. The switch to your own gateways is a separate step, and the console will not let you make it until at least one of your gateways has connected, so you cannot cut yourself off by accident.

Switching back is the same step in reverse.