68 min readMehdi Hadeli

Running MCP Servers in Production: agentgateway + ToolHive, from stdio to a Multiplexed HTTP Endpoint

On this page

Table of contents

Introduction

The Model Context Protocol (MCP) gives AI assistants a standard way to connect to tools, data, and memory. Running an MCP server is only half the job. As soon as it can fetch a URL, read a knowledge graph, or start another tool, you need authentication (who is calling?) and authorization (what may they do?). MCP runs over stdio or HTTP, but the protocol leaves those security decisions to the deployment.

We will build one self-contained MCP gateway sample: four Docker-based MCP servers (memory, fetch, thinking, and a federated /mcp view) behind agentgateway. The gateway uses four auth mechanisms:

  1. Context7-style per-user API keys — static keys, one per user, with per-user quotas and attribution.
  2. mcpAuthentication — OAuth 2.0 resource-server protection per the MCP Authorization specification, backed by Keycloak.
  3. mcpAuthorization — per-tool CEL rules that filter denied tools out of the client's view entirely.
  4. oidc browser authenticationKeycloak-backed encrypted-cookie sessions that secure the gateway's own Admin UI and any browser-facing web route.

Everything runs in Docker. I also checked the claims against the running stack: 401s, 429s, filtered tool lists, and trace attributes. The setup exposed a few practical problems along the way, including Keycloak's UUID sub, dynamic client registration returning 500, trailing-slash redirect mismatches, and unknown scopes breaking the browser flow.

Goal. By the end you'll understand when to pick each auth mechanism, how to wire them into agentgateway, and what to watch out for — with the official docs linked at every step.

Related reading. For a broader gateway that adds LLM routing, guardrails, A2A, OpenAPI-to-MCP, and full observability, see Building an AI Gateway with AgentGateway: LLM, MCP, and A2A Behind One Door.

Why put a gateway in front of your MCPs at all?

MCP servers already work, so why put a gateway in front of them? Once you have several servers, real users, or a deployment environment to operate, the raw setup becomes difficult to manage. Six practical pressures point toward a gateway.

1. Multiple stdio servers need one HTTP surface

Most MCP servers today ship as stdio processes: you launch docker run -i mcp/memory or npx @modelcontextprotocol/server-fetch and talk to it over stdin/stdout. That's great for a single local tool, and it's exactly how IDE integrations like VS Code Copilot spawn servers. But it becomes a problem when you have several:

# three servers, three different ways to run them
docker run -i mcp/memory               # memory (persisted graph)
docker run -i mcp/fetch                # fetch a URL
npx -y @modelcontextprotocol/server-everything   # everything server

Every client needs to know how to launch each one: the container image, npx package, environment, and startup flags. With a fleet of 10 servers, every agent, IDE, and script duplicates that knowledge. stdio also requires the client to share a filesystem or process boundary with the server, so it cannot serve clients on other machines or scale horizontally.

A gateway changes the boundary. The servers stay as they are, still stdio processes in containers, while the gateway exposes them as HTTP endpoints with subpaths. One URL can reach many servers:

http://gateway:18080/memory    → docker run -i mcp/memory
http://gateway:18080/fetch     → docker run -i mcp/fetch
http://gateway:18080/thinking  → docker run -i mcp/sequentialthinking

Clients get one base URL, one port, and one credential model. They connect over plain HTTP or SSE without knowing how a backend is packaged. The gateway translates HTTP to stdio. This is the "unified HTTP with subpath routing" pattern and the core job of an MCP gateway like agentgateway.

2. Security: one place to enforce who can do what

Without a gateway, every MCP server must implement its own auth — and most don't. The typical MCP server ships with no authentication at all: it assumes a trusted local caller. If you expose such a server directly to a network (or to an agent with network access), every tool it exposes — fetch, file access, memory writes — becomes available to anyone who can reach it. That's not a hypothetical: the fetch server will happily retrieve arbitrary URLs, and a memory server will happily mutate its graph, for whoever asks.

A gateway gives you a single enforcement point for three security concerns:

  • Authentication (who?) — every request is checked before it ever reaches an MCP server: API key, OAuth token, or session cookie, per route. Servers stay dumb and unauthenticated behind the gateway; the gateway is the only thing exposed.
  • Authorization (what may they do?) — not just "is this user real" but "which tools may this user call". Per-tool rules (mcpAuthorization) can even hide denied tools from the client entirely.
  • Observability (what happened?) — one place to log, trace, rate-limit, and attribute every call to a user, which is impossible if each server logs independently.

Add per-user quotas and revocation (disable the user, delete the key) and you have a security story you could never bolt onto N independent servers.

3. Unified MCPs: federation instead of fragmentation

The third reason is the one users feel every day: too many servers = context overload. Connect memory, fetch, thinking, and a couple of SaaS tools to your IDE and you get a dozen separate tool lists to scroll through, each with its own namespace and its own auth dance. Worse, different teams standardize on different servers, so collaboration means "install these six servers and configure them the same way I did."

A gateway can federate: expose all backends under one endpoint, with tool names namespaced so there are no collisions:

/mcp → memory_create_entities, memory_read_graph,     # from mcp/memory
       fetch_fetch,                                    # from mcp/fetch
       thinking_sequentialthinking                     # from mcp/sequentialthinking

One server entry in the client, one tool list, one auth flow — while the backends stay independently deployable and swappable. Swap mcp/fetch for a different fetch implementation and the client-facing surface doesn't change. That's the "unified MCP" payoff: you keep the modularity of many small servers and the ergonomics of one.


4. Remote access: stdio is local-only, HTTP is anywhere

stdio transport has a hard constraint: the client and server must share a process boundary (or at least a filesystem + spawn mechanism). That's fine for a tool on your laptop, and it's exactly why VS Code Copilot can docker run a server locally. But it means no remote clients, no sharing, no central deployment:

  • Your teammate on another machine can't reach the memory server running on your box.
  • A server deployed in a Kubernetes cluster can't be reached by agents outside the pod.
  • Every client machine needs the server image, the runtime, and the config — duplicated everywhere.

Exposing the MCPs over HTTP through a gateway removes that constraint: any client that speaks HTTP (an agent, a web app, a teammate's IDE pointed at a shared URL) can reach the same servers. The gateway is the only thing that needs to be reachable — the backends stay in containers, isolated, spawnable per request. And HTTP transport is what unlocks the OAuth flows in this article: bearer tokens, PKCE, refresh — none of which have a meaning over raw stdio.

5. Operational lifecycle: one owner for spawning, swapping, and health

With N raw stdio servers, who owns the process lifecycle is ambiguous. If the client spawns them (docker run), the client also decides when they restart, what version runs, and what to do when one crashes. Multiply by every consumer and you get version drift, orphaned containers, and no single source of truth for "what's deployed."

A gateway centralizes the lifecycle:

  • Spawn on demand — the gateway either forks the stdio servers itself (native stdio host binary) or a host-side runtime (ToolHive's thv daemon, or the sample's lighter mcpwrap Go wrapper) launches a container per server on first use and restarts it on crash; the gateway never spawns containers and holds no Docker access.
  • Swap without touching clients — replace the mcp/fetch image with a new implementation in the gateway config; the client-facing URL and tool names don't change. Same URL, new backend — that's the "swappability" a gateway buys you for free.
  • Health and readiness — the gateway exposes admin/readiness endpoints (/healthz, metrics) so your monitoring watches one service instead of N ad-hoc processes.

6. Centralized configuration and cost control

Finally, a gateway is the one place where policy lives in one file — and policy includes money. Agents can hammer tools hundreds of times per minute; each call may carry real cost (fetching URLs, running model-backed tools, mutating shared state). Without a central choke point, there's no way to answer:

  • How many requests is user X making per minute? — per-user rate limits (localRateLimit).
  • What happens if a runaway loop hits the fetch server 10,000 times? — token-bucket quotas return 429 instead of unbounded cost.
  • Who is consuming what? — request-log database + per-user attribution in traces/metrics.

The same config file that defines routes and auth also defines quotas, logging fields, and standard attributes — one artifact to review, one place to audit, no policy scattered across N servers.


Those six pressures are the reason for this sample: one HTTP surface, centralized security, federation, remote access, lifecycle ownership, and cost control. The rest of the article focuses on security and verifies each mechanism against a running gateway.

Architecture Overview

The sample uses agentgateway's standalone model. Three runtimes are provided for the same gateway config — pick one, run one at a time (an optional fourth variant, Approach 4, drops the gateway entirely and lets ToolHive's own vMCP aggregate + OIDC-protect):

  1. Native stdio (host binary) — the gateway itself spawns each MCP server as a host subprocess (stdio: { cmd, args }), the documented flow from the connect/stdio docs. No bridge, no custom image — the gateway runs as a host binary.
  2. ToolHive (thv) — the production default: a host-side MCP runtime with auto-restart, health monitoring, optional egress/DNS guardrails for untrusted servers, and npx:///uvx:// protocol schemes that can host npm-only servers (like everything) by building a container on demand; the gateway stays the stock container image.
  3. mcpwrap (Go wrapper) — a ~300-line Go CLI, kept for teaching (it shows the stdio→HTTP bridge ToolHive hides); runs the three official images as plain stdio containers on the host; docker-images-only, no production lifecycle.

Each variant ships its own compose file, config, and start/stop scripts (deployments/docker-compose.stdio.yml/deployments/config.stdio.yaml/scripts/start-stdio.sh, deployments/docker-compose.toolhive.yml/deployments/config.toolhive.yaml/scripts/start-toolhive.sh, deployments/docker-compose.mcpwrap.yml/deployments/config.mcpwrap.yaml/scripts/start-mcpwrap.sh). The auth story — API keys, OAuth, CEL authorization, quotas — is identical in all three, because the gateway only ever sees plain HTTP (or spawns the processes itself in approach 1). Two ports expose the same four routes with different auth:

Sample code for every approach is available in one placesamples/mcp-gateway-agentgateway/ in this repository. It ships everything this article walks through: the per-variant compose files (docker-compose.stdio.yml, docker-compose.toolhive.yml, docker-compose.mcpwrap.yml, docker-compose.vmcp.yml), the gateway configs (config.stdio.yaml, config.toolhive.yaml, config.mcpwrap.yaml, vmcp.yaml), the Keycloak realm export, the observability stack (Grafana dashboards, Tempo, Prometheus, Loki, otel-collector), the start/stop scripts, and the full mcpwrap Go source — so every snippet in the article below is runnable as-is, one approach at a time.

PortAuth mechanismCredential
:18080API key (Context7-style)x-api-key: sk-alice-demo-key (per user)
:8082mcpAuthentication (OAuth 2.1 / MCP auth spec)Keycloak-issued JWT (per user)
Rendering diagram...

Key architectural facts:

  • Route-level policies. agentgateway attaches auth to routes, not just gateways. This is what lets the SSO port carry mcpAuthentication + mcpAuthorization + per-user rate limits on the same route, with JWT claims flowing into every policy. See the routes documentation.
  • Two gateways, one config. default (apiKey) and sso (OAuth) are both defined in the same config file (e.g. deployments/config.toolhive.yaml) — see the gateways reference.
  • Host-side runtime, hardened gateway. In the container variants the gateway container is the stock agentgateway image — no Docker CLI, no docker.sock, non-root, read-only rootfs, cap_drop: ALL. MCP servers run on the host as stdio containers bridged stdio→HTTP by a host runtime (ToolHive's thv in this diagram — the production choice; the mcpwrap wrapper in Approach 3 is the same bridge, kept for teaching). In the native stdio variant the gateway is a host binary that forks the servers itself — no gateway container, no custom image, but also no container isolation (see the three approaches). The vMCP variant (Approach 4) is a fourth shape: no agentgateway at all — thv vmcp serve aggregates the four servers on :4483/mcp and OIDC-protects it with the same Keycloak realm.

Multiplexing (Virtual MCP) — one endpoint, all tools

The first three routes expose one server per path. The fourth, /mcp, is where multiplexing kicks in: one endpoint that federates every backend, so an end-user client (agent, IDE, script) configures a single connection string and sees every tool from every MCP server.

Per the docs, multiplexing is a property of putting several targets in one backend — it is not a feature of the top-level mcp: section. Both forms produce the same MCP backend; the choice between them is covered in MCP configuration modes. This sample actually uses both: the routing-based /mcp route below (4 targets in the stdio and ToolHive variants), and the top-level mcp: block on the sso gateway (:8082), protected by the same Keycloak OAuth — same idea, two shapes. The four stdio targets are the official reference servers from the modelcontextprotocol/servers repo — memory, fetch, sequentialthinking, and everything (a kitchen-sink test server exposing ~13 tools incl. echo; ships as an npm package only — no docker image — so only the stdio and ToolHive variants host it, via npx / npx://) — all four listed as ready under MCP → Servers in the Admin UI.

# deployments/config.stdio.yaml — the /mcp route: FOUR targets in ONE backend
- name: multiplexing-mcp-apikey
  gateways: [default]
  matches:
    - path: { pathPrefix: /mcp }
  backends:
    - mcp:
        targets:
          - name: memory
            stdio:
              cmd: npx
              args: ['-y', '@modelcontextprotocol/server-memory']
          - name: fetch
            stdio:
              cmd: uvx
              args: ['--with', 'mcp<2', 'mcp-server-fetch']
          - name: thinking
            stdio:
              cmd: npx
              args: ['-y', '@modelcontextprotocol/server-sequential-thinking']
          - name: everything
            stdio:
              cmd: npx
              args: ['-y', '@modelcontextprotocol/server-everything']

A client that connects to /mcp receives a single virtual MCP server whose tools/list response is the union of every target's tools — namespaced by target name so identical tool names never collide (memory_create_entities, fetch_fetch, thinking_sequentialthinking). The default prefixMode (conditional) prefixes only when a backend has more than one target; the options are described in Tool name prefixing. This is federation, not load balancing — every request fans out to the right target by name; it is not picking one of several identical backends (multiplexing vs. load balancing).

The end-user client only ever configures ONE endpoint. Verified live in this sample:

Client endpointAuthTools seen (verified)
http://localhost:18080/mcpx-api-key (per user)24 — memory (9) + fetch (1) + thinking (1) + everything (~13)
http://localhost:8082/mcpKeycloak JWT (per user)alice 24, bob 23 — fetch_fetch hidden by mcpAuthorization (top-level mcp: block: 4 targets incl. everything)

The Admin UI Tool Playground shows the same federated view: after Initialize it lists all 24 namespaced tools and calls e.g. everything_echo straight through the virtual server. (The mcpwrap variant hosts 3 targets only — 11 tools on /mcp.)

Multiplexing works across every transport. A target is just an entry in a backend; each target may use a different connection method and the federation is unchanged — so a local stdio server, a remote streamable-HTTP MCP, and a legacy REST API can all sit behind the same /mcp endpoint:

Target typeDocsWhat it connects toExample config
stdioconnect/stdioa command the gateway spawns locally (this sample)cmd: npx, args: ["-y", "@modelcontextprotocol/server-memory"]
mcp (streamable HTTP)connect/httpa remote MCP server over streamable HTTP (stateful sessions pinned per request)host: https://mcp.example.com/v1/mcp
openapiconnect/openapia REST API described by an OpenAPI spec — one MCP tool per operation (operationId)openapi: { schema: { url: ... }, host: ... }

That's the "unified MCP" payoff from the introduction, made concrete: keep the modularity of many small servers, give clients the ergonomics of one.

Approach 1 — the gateway spawns the servers itself (native stdio)

The most agentgateway-native option, and the one shown in the official connect/stdio docs: the gateway process forks each MCP server as a stdio subprocess and speaks JSON-RPC over stdin/stdout. No bridge runtime, no HTTP proxy, no extra containers — the stdio target is the transport:

# deployments/config.stdio.yaml
backends:
  - mcp:
      targets:
        - name: memory
          stdio:
            cmd: npx
            args: ['-y', '@modelcontextprotocol/server-memory']
        - name: fetch
          stdio:
            cmd: uvx
            args: ['--with', 'mcp<2', 'mcp-server-fetch']
        - name: thinking
          stdio:
            cmd: npx
            args: ['-y', '@modelcontextprotocol/server-sequential-thinking']
        - name: everything
          stdio:
            cmd: npx
            args: ['-y', '@modelcontextprotocol/server-everything']

The catch: stdio targets run where the gateway process runs. The stock cr.agentgateway.dev/agentgateway container image is a single static binary — no Node, no Python — so it cannot execute npx or uvx. The documented flow for stdio servers is therefore the host binary: install agentgateway on the host (curl -sL https://agentgateway.dev/install | bash) and run agentgateway -f deployments/config.stdio.yaml. That is exactly what this variant does — no custom Dockerfile, no custom image (the stock container image stays untouched; it's simply not used as the gateway in this variant).

host:  agentgateway -f deployments/config.stdio.yaml   (:18080 apiKey, :8082 SSO, :15000 UI, :15020 metrics)
         ├─ npx @modelcontextprotocol/server-memory          (stdio subprocess)
         ├─ uvx --with "mcp<2" mcp-server-fetch            (stdio subprocess)
         ├─ npx @modelcontextprotocol/server-sequential-thinking (stdio subprocess)
         └─ npx @modelcontextprotocol/server-everything          (stdio subprocess)
compose (deployments/docker-compose.stdio.yml): keycloak + otel-collector + prometheus + tempo + loki + grafana + langfuse + phoenix — NO gateway service

deployments/docker-compose.stdio.yml runs infrastructure only (Keycloak + observability); the gateway and its server subprocesses live on the host. scripts/start-stdio.sh installs the binary and ensures npx (Node) + uvx (via pip install uv if missing) — the packages are fetched on demand — brings up the compose infra, and starts the gateway detached (pid in logs/agentgateway-stdio.pid); scripts/stop-stdio.sh tears it all down.

Because the gateway is a host process, a few config details differ from the container variants: the apiKey gateway binds :18080 directly (Keycloak owns 8080 on the host), OTLP goes to http://localhost:4317 (the collector's published port, not the in-network name), the request-log SQLite lives at a host-writable path (/tmp/agentgateway/), and Prometheus scrapes the host binary via extra_hosts: ["gateway:host-gateway"].

Verifying through the Admin UI playground. The UI's MCP > Tool Playground connects to the top-level mcp: section, not to per-route backends[].mcp targets. In deployments/config.stdio.yaml that section lives on the sso gateway :8082 and federates the same four stdio targets into one virtual server (served at /mcp and /sse), protected by the same Keycloak OAuth as the SSO routes. To verify access to tools from the browser, follow the official flow: open http://localhost:15000/ui/mcp/playground, click Apply CORS if the browser-origin notice appears, expand Authorization header and paste a Keycloak JWT for alice as the Bearer token — mint one at the realm's token endpoint:

curl -s -X POST http://localhost:8081/realms/mcp-demo/protocol/openid-connect/token \
  -d "grant_type=password&client_id=mcp-gateway&username=alice&password=alice123"
# → take the access_token value

then click Initialize — the playground lists all 24 federated tools (memory_*, fetch_fetch, thinking_sequentialthinking, everything_*). Select a tool (e.g. everything_echo), type a message, and Call tool to see the echoed result — verified live:

Echo: SSO verified through the UI!

agentgateway Admin UI — native stdio approach, MCP Servers page showing 4 Command Line targets ready

agentgateway Admin UI — native stdio approach, Tool Playground initialized with 24 federated tools

Two config details make this work: the cors policy inside the mcp: block (allowOrigins includes http://localhost:15000; strict mcpAuthentication would otherwise 401 the browser's CORS preflight before the policy layer can respond) and the mcpAuthentication policy (issuer/audiences/jwks pointing at Keycloak) so the playground's Bearer token field validates the JWT. The per-route backends on :18080/:8082 remain the primary API for programmatic clients — the top-level mcp: section is the docs-style simplified form that the UI understands (both forms produce the same MCP backend; see configuration modes).

Advantages (summary: zero moving parts, gateway-native, tiny footprint)

  • Zero extra moving parts — one binary, no host daemon, no proxies, no custom image, nothing to build.
  • Fully agentgateway-native — the gateway owns server sessions, process lifecycle, and health; no third-party runtime in the path.
  • Smallest host footprint — no wrapper binary, no thv state, no proxy ports to secure.

Disadvantages (summary: shared fate, host prerequisites)

  • Not containerized — every server shares the host OS: no --network none, no egress guardrails, no per-server isolation.
  • Shared fate — a crashing server subprocess can destabilize the gateway process itself.
  • Host prerequisites — the host must carry Node/npx + uv, and PATH must resolve npx / uvx for the gateway's spawned processes.
  • Gateway logs to a file — no Docker log driver, so Loki collects the infra containers only (logs/agentgateway-stdio.log for the gateway).

Best for: the most agentgateway-native setup on a single trusted host where installing the server runtimes is fine and container isolation is not a requirement.

Approach 2 — a runtime hosts the stdio servers (ToolHive, the production default)

This is the sample's production-recommended deployment model: instead of giving the gateway container Docker privileges (docker.sock + the Docker CLI, running as root), a runtime on the host — ToolHive — owns the MCP containers, and agentgateway routes to ToolHive's HTTP proxies as plain mcp.host targets. The entire auth story in this article does not change — same config, same routes, same Keycloak — only the backend addresses differ (deployments/config.toolhive.yaml targets http://host.docker.internal:19002/mcp instead of spawning docker run children). The gateway stack stays a single hardened container with zero Docker privileges.

ToolHive runs each server as a container and exposes its own streamable-HTTP proxy per workload; run MCP servers with ToolHive is the reference for everything below. The four workloads in this sample:

thv run docker.io/mcp/memory             --host 0.0.0.0 --proxy-port 19001 --transport stdio --proxy-mode streamable-http --isolate-network=false
thv run docker.io/mcp/fetch              --host 0.0.0.0 --proxy-port 19002 --transport stdio --proxy-mode streamable-http
thv run docker.io/mcp/sequentialthinking --host 0.0.0.0 --proxy-port 19003 --transport stdio --proxy-mode streamable-http --isolate-network=false
# everything ships as an npm package only (no docker image) — ToolHive's
# npx:// protocol scheme builds a container from the package on demand:
thv run npx://@modelcontextprotocol/server-everything \
  --name mcp-everything --host 0.0.0.0 --proxy-port 19004 \
  --transport stdio --proxy-mode streamable-http --isolate-network=false

That last line is the key advantage over the other two approaches: ToolHive hosts servers that have no docker image at all, by wrapping the npm/pip package in a container (npx:///uvx:// schemes) — the one thing mcpwrap (Approach 3) structurally cannot do.

How a stdio-only image becomes an HTTP endpoint

The official images expose no HTTP port — they speak newline-delimited JSON-RPC over stdin/stdout. ToolHive bridges that with two processes (transport architecture):

  1. The container runs the stock image with stdin attached (AttachStdin/AttachStdout) and zero port bindings.
  2. A host-side proxy process owns the HTTP port, reads JSON-RPC from the container's stdout, and writes requests back into its stdin — Container Attach (Stdio Transport).

Two flags make this work (thv run CLI reference):

  • --transport stdio — tells ToolHive the image is a stdio server, so it attaches stdin instead of assuming the container self-hosts HTTP. Without it, stdio-only servers die instantly (exit 0 → restart loop → proxy 502).
  • --proxy-mode streamable-http — the HTTP shape exposed to clients (streamable HTTP per the MCP spec); the container never sees it.

One container per server when network isn't needed

Default network isolation (--isolate-network=true) gives every workload its own internal bridge plus egress (Squid) and DNS (dnsmasq) sidecars that enforce the permission profile's outbound rules (RunConfig and Permission Profiles). That's 3 containers per workload, even for servers that never touch the network. Isolation governs outbound traffic only — the stdio bridge is unaffected. So the rule we apply:

Drop isolation (--isolate-network=false) for any server that needs no outbound network: one container per server, same behavior.

In this sample that's mcp/memory and mcp/sequentialthinking (1 container each), while mcp/fetch keeps isolation because it fetches arbitrary URLs (3 containers: fetch + fetch-egress + fetch-dns), and mcp-everything — a test server whose echo/sample tools don't need outbound access — also drops isolation (1 container, built on demand from the npm package). 7 host containers instead of 12, and the gateway stack stays a single hardened container with zero Docker privileges:

Rendering diagram...

The runnable ToolHive variant lives in samples/mcp-gateway-agentgateway/ as deployments/docker-compose.toolhive.yml + deployments/config.toolhive.yaml, with the exact thv run commands in the README. A one-liner brings the host workloads up idempotently after a reboot — ./scripts/start-toolhive.sh (the thv proxies are host processes, so they do not survive a terminal close; the script cleans the stale state and restarts them without touching the gateway).

agentgateway Admin UI — ToolHive approach, MCP Servers page showing 4 Streamable HTTP targets ready

agentgateway Admin UI — ToolHive approach, Tool Playground initialized with 24 federated tools

Approach 3 — the mcpwrap wrapper (teaching only)

The same sample also ships a second runtime — mcpwrap, a ~300-line Go CLI (installable with go install) that implements the very same two-step bridge described above: it runs each official image as a plain stdio container (docker run -i --rm --name mcpwrap-<name>) and proxies the newline-delimited JSON-RPC to Streamable HTTP, matching responses by JSON-RPC id and minting an Mcp-Session-Id on initialize. The gateway config is identical except the backend addresses (deployments/config.mcpwrap.yaml targets host.docker.internal:19101-19103/mcp):

./scripts/start-mcpwrap.sh   # build wrapper -> mcpwrap daemon (:19101-19103) -> loki driver -> compose stack
./scripts/stop-mcpwrap.sh    # stop everything

The wrapper makes the transport mechanics visible and keeps the host dependency-free: no extra runtime to install, no permission profiles, no egress/DNS sidecars — servers that never open outbound connections run with docker run --network none, servers that do (fetch) keep the default bridge. That is 3 host containers total, and the whole bridge fits in one readable proxy.go. This is exactly the machinery ToolHive hides behind thv run — which is why mcpwrap is kept as a teaching tool, not a production runtime.

Why not production? Two hard limits. First, docker-images-only: there is no npx:///uvx:// support, so mcpwrap cannot host npm-only servers like everything (in this sample the mcpwrap variant hosts 3 servers, not 4, and /mcp exposes 11 tools instead of 24). Second, no lifecycle: no auto-restart, no health-based recovery, no egress guardrails. For a demo or a locked-down host it's the sweet spot; for anything real, use Approach 2.

agentgateway Admin UI — mcpwrap approach, MCP Servers page showing 3 Streamable HTTP targets ready

agentgateway Admin UI — mcpwrap approach, Tool Playground initialized with 11 federated tools

Rendering diagram...

Both runtimes share the same compose observability stack and the same scripts/ launchers pattern (start/stop pairs); run one at a time, since the stacks publish the same host ports. Everything else in this article — API keys, OAuth, CEL authorization, quotas — behaves identically on either runtime, because the gateway only ever sees plain HTTP.

Which runtime should you use — the three approaches compared

agentgateway talks to its backends over streamable HTTP (mcp.host targets) or spawns a server itself (stdio: { cmd, args }). The real question is: who converts stdio ↔ HTTP, and where do the server processes live? The sample ships all three answers with separate compose files, configs, and scripts:

1. native stdio (host binary)2. ToolHive (thv) — recommended3. mcpwrap (Go wrapper, teaching)
Gatewayhost binary, no containerstock container imagestock container image
Bridge runtimenone — the gateway forks the serversthv daemon (full MCP runtime)~300-line Go CLI (go install)
Server processessubprocesses of the gateway (shared OS)7 containers (fetch + egress Squid + DNS sidecars; everything via npx://)3 stdio containers, no sidecars
Container isolationnone — all share the host OSegress proxy + DNS sidecars (SSRF guard) for untrusted servers--network none for no-outbound servers
Server lifecyclegateway-managed (restart = gateway restart)auto-restart, health, crash backoffpull/run/stop, no auto-restart
Crash isolationa server crash can take the gateway downisolated per containerisolated per container
Host prerequisitesNode/npx + uvDocker + thv CLIDocker + ~10 MB exe
Gateway logsfile (logs/agentgateway-stdio.log)container logs → Loki drivercontainer logs → Loki driver
npm-only servers (npx:///uvx://, e.g. everything)✅ (npx on the host)✅ (npx:// builds a container on demand)❌ docker images only
Best forzero-dependency, single trusted host, most agentgateway-nativeproduction default — lifecycle, fleet, untrusted serversdemo / teaching the stdio→HTTP bridge

Which one is preferred with agentgateway?

  • Approach 2 (ToolHive) is the production default. It keeps the hardened stock-image gateway, isolates every server in its own container, adds real workload lifecycle (auto-restart, health, crash backoff) and egress guardrails (Squid/DNS sidecars) for untrusted servers, and — uniquely — hosts npm-only servers (everything) via its npx:///uvx:// protocol schemes (see run MCP servers with ToolHive). The cost: a daemon + sidecars on the host.
  • Approach 1 (native stdio) is the most agentgateway-native: no extra runtime, the gateway owns the server processes and their sessions, and it needs only a host binary — no custom Dockerfile. The cost: zero container isolation (a crash can take the gateway with it) and you must install the server runtimes on the host.
  • Approach 3 (mcpwrap) is the teaching tool: identical routes/auth/policies with 3 containers instead of 7 and the whole bridge readable in one proxy.go — the demo that makes ToolHive's internals visible. Not for production: docker-images-only, no lifecycle.
  • No bridge at all whenever the server already speaks HTTP: point mcp.host straight at it and skip all three. stdio bridging is only needed for stdio-only servers in front of a hardened gateway.

Approach 4 (optional) — ToolHive's own gateway: Virtual MCP Server (vMCP)

So far the gateway is always agentgateway. But ToolHive itself ships a gateway layer — the Virtual MCP Server (vMCP) — that aggregates the MCP servers of a ToolHive group into one unified endpoint and can OIDC-protect it with the very same Keycloak realm this sample uses. The sample implements it as Approach 4 (all files under samples/mcp-gateway-agentgateway/: vmcp.yaml, docker-compose.vmcp.yml, scripts/start-vmcp.sh and scripts/stop-vmcp.sh) so the comparison is concrete: run it and see what a real AI gateway like agentgateway adds on top.

Two deployment shapes. vMCP exists as a Kubernetes operator resource (VirtualMCPServer CRD, prod) and as a local CLI (thv vmcp serve, dev) — the sample uses the CLI:

# 1) same 4 workloads as Approach 2, but in a group the vMCP reads from
thv group create mcp-vmcp
thv run docker.io/mcp/memory             --group mcp-vmcp --host 0.0.0.0 --proxy-port 19011 --transport stdio --proxy-mode streamable-http --isolate-network=false
thv run docker.io/mcp/fetch              --group mcp-vmcp --host 0.0.0.0 --proxy-port 19012 --transport stdio --proxy-mode streamable-http
thv run docker.io/mcp/sequentialthinking --group mcp-vmcp --host 0.0.0.0 --proxy-port 19013 --transport stdio --proxy-mode streamable-http --isolate-network=false
thv run npx://@modelcontextprotocol/server-everything --name mcp-everything \
  --group mcp-vmcp --host 0.0.0.0 --proxy-port 19014 \
  --transport stdio --proxy-mode streamable-http --isolate-network=false

# 2) aggregate + OIDC-protect them on ONE endpoint
thv vmcp serve --config deployments/vmcp.yaml --host 0.0.0.0 --port 4483
#    -> http://127.0.0.1:4483/mcp  (all 4 servers, tools prefixed mcp-<workload>_)

deployments/vmcp.yaml (the ground truth was captured with thv vmcp init --group mcp-vmcp and then patched) mirrors the agentgateway concepts one-to-one:

name: mcp-vmcp
groupRef: mcp-vmcp # the ToolHive group (operational visibility)
incomingAuth: # = agentgateway mcpAuthentication
  type: oidc
  oidc:
    issuer: http://keycloak:8080/realms/mcp-demo # same Keycloak realm
    clientId: mcp-gateway # same public PKCE client
    audience: mcp-gateway # tokens must carry aud=mcp-gateway
    resource: http://127.0.0.1:4483/mcp # RFC 9728 resource — unlocks PRM metadata
    insecureAllowHttp: true # dev-only: Keycloak is plain HTTP
    jwksAllowPrivateIp: true # dev-only: keycloak resolves to 127.0.0.1
outgoingAuth:
  source: inline
  default: { type: unauthenticated } # backends are plain localhost proxies
aggregation: # = agentgateway /mcp tool-name prefixing
  conflictResolution: prefix
  conflictResolutionConfig:
    prefixFormat: '{workload}_'
backends: # STATIC list — vMCP uses exactly these
  - { name: mcp-memory, url: http://127.0.0.1:19011/mcp, transport: streamable-http }
  - { name: mcp-fetch, url: http://127.0.0.1:19012/mcp, transport: streamable-http }
  - { name: mcp-sequentialthinking, url: http://127.0.0.1:19013/mcp, transport: streamable-http }
  - { name: mcp-everything, url: http://127.0.0.1:19014/mcp, transport: streamable-http }

Verified live (aggregation smoke-test with incomingAuth: anonymous, then the OIDC config re-enabled): initialize + tools/list on http://127.0.0.1:4483/mcp returns 24 tools, namespaced exactly like agentgateway's federated /mcpmcp-memory_create_entities, mcp-fetch_fetch, mcp-sequentialthinking_sequentialthinking, mcp-everything_echo, … (9 + 1 + 1 + 13). Without a Bearer token the OIDC middleware returns 401 invalid_token. One connection string, all tools — the same multiplexing payoff, implemented by ToolHive instead of agentgateway.

PKCE / Keycloak integration. The same realm, client, and users work unchanged — the vMCP validates the exact same JWTs agentgateway's :8082 accepts (issuer http://keycloak:8080/realms/mcp-demo, audience mcp-gateway, live JWKS via OIDC discovery). The browser/PKCE flow is identical to the playground flow: clients discover the resource's OAuth metadata and get tokens from Keycloak directly. One difference worth knowing: agentgateway proxies Keycloak's AS metadata and dynamic client registration on its own well-known paths (mock 201 DCR), whereas the vMCP CLI serves the RFC 9728 protected resource metadata (GET /.well-known/oauth-protected-resource{"resource": "http://127.0.0.1:4483/mcp", "authorization_servers": ["http://keycloak:8080/realms/mcp-demo"], "bearer_methods_supported": ["header"], "scopes_supported": ["openid"]} — verified live) and points clients at Keycloak for AS discovery/DCR. In the operator (K8s) shape, vMCP instead runs its own embedded authorization server with real DCR (RFC 7591) and CIMD and mints its own JWTs — that is the deployment where vMCP fully short-circuits an external IdP.

Two gotchas surfaced while verifying this live — both fixed in the sample (deployments/keycloak/realm-export.json + deployments/vmcp.yaml):

  • Keycloak 26 did not emit a sub claim. The sample realm's client had only a preferred-username mapper, so password-grant tokens carried preferred_username but no sub — and vMCP's identity layer hard- requires sub (OIDC Core 5.1), failing every request with 401 invalid_token: Invalid authentication claims. The realm export now adds a sub protocol mapper (oidc-sub-mapper). agentgateway never complained because it keys on preferred_username; the vMCP is stricter.
  • No aud claim without an audience mapper. With audience: mcp-gateway configured, tokens lacking aud fail invalid audience. The export adds an oidc-audience-mapper (audience-mcp-gateway) so the same realm works for both gateways.
  • resource unlocks the discovery endpoint. Without it the CLI vMCP returns 404 on /.well-known/oauth-protected-resource (the handler refuses to presume a resource URL); deployments/vmcp.yaml sets resource: http://127.0.0.1:4483/mcp and the metadata above is served.

The API-key gap. agentgateway gives you two credential models: per-user API keys (:18080) and OAuth (:8082). The vMCP CLI gives you only OIDC (or anonymous) — there is no incoming API-key authentication in incomingAuth. If you need Context7-style keys, put a gateway (any API gateway, or agentgateway) in front of the vMCP. What vMCP offers instead is outgoing auth to the backends — static header injection and passthrough headers (also useful behind a gateway that terminates auth):

outgoingAuth:
  source: inline
  backends:
    mcp-myapi: # backend name
      type: headerInjection
      headerInjection:
        headers:
          Authorization: 'Bearer <static-upstream-key>'
    # or forward the client's own credential to the backend:
  default:
    type: passthroughHeaders
    passthroughHeaders: ['Authorization']

vMCP vs agentgateway — the feature comparison. Where the two overlap they implement the same MCP auth spec; where they differ, each has things the other lacks:

FeaturevMCP (ToolHive)agentgateway
Multiplexing (one endpoint, all tools)Yes — aggregation.conflictResolution: prefixYes — /mcp route + top-level mcp: block
OAuth/PKCE (MCP auth spec, RFC 9728)Yes — OIDC JWT validation, PRM metadataYes — mcpAuthentication + proxied AS metadata/DCR
DCRK8s: real DCR (RFC 7591) + CIMD; CLI: client goes to Keycloakmock 201 short-circuit (no initial-access token)
Incoming API keysNo (oidc/anonymous only)Yes — per-user x-api-key (:18080)
Browser OIDC cookies (web routes, Admin UI)NoYes — oidc mechanism (:15000/ui, Mechanism 4)
Admin UI / tool playgroundNo (CLI + /status endpoint)Yes — http://localhost:15000/ui
Per-route authz + rate limitspartial (Cedar authz; core rate limiter)Yes — CEL mcpAuthorization + localRateLimit per route
LLM gateway (providers, fallbacks, model routing)NoYes — full AI gateway
Cedar policy authorizationYes (incomingAuth.authz)No (uses CEL)
Tool optimizer (find_tool/call_tool) + composite toolsYesNo
Audit loggingYes (--enable-audit)Yes — request-log DB + traces
Kubernetes-nativeYes VirtualMCPServer CRDagentgateway CRDs (separate operator)

Should you use it — and should you combine it with agentgateway?

  • vMCP alone is the right call when all you need is pure MCP aggregation + OAuth for a small team: fewer moving parts than a full gateway, one binary (thv vmcp serve) or one CRD, and it reuses your existing IdP. The gaps — API keys, Admin UI, LLM-gateway features — only matter if you actually need them.
  • agentgateway in front of the ToolHive runtime (Approach 2) is the choice when you want the full AI gateway on top of the same workloads: per-user keys, browser OIDC, CEL authz + rate limits, LLM provider management, Admin UI, and richer observability. Here the vMCP is redundant — the gateway already multiplexes and OIDC-protects.
  • vMCP behind agentgateway only pays off if you specifically want its Cedar authz, tool optimizer, or composite tools applied downstream — otherwise it duplicates what the gateway already does.

Verdict: the two are complementary, not competing — vMCP is ToolHive's answer to "gateway needed, but only for MCP"; agentgateway is the answer when the gateway should also be an AI gateway (LLM routing, quota enforcement with per-user keys, UI, observability). This article's sample keeps both: approaches 1-3 cover the agentgateway side, Approach 4 covers the vMCP side, all sharing one Keycloak realm so a single minted token works everywhere.

How to host any MCP server (packaging → choice)

The deciding question is how the MCP server is packaged. The matrix, with the exact sample servers mapped:

Server packagingApproach 1 (stdio)Approach 2 (ToolHive)Approach 3 (mcpwrap)Notes
Docker image (mcp/memory, mcp/fetch, …)cmd: npx/uvx (if also on npm/PyPI)thv run docker.io/mcp/fetchdocker run -i --rmall three run official images
npm package only (server-everything)cmd: npx, args: [-y, …]thv run npx://@modelcontextprotocol/server-everything❌ no npx:// supportthe everything server here — mcpwrap cannot host it
PyPI/uvx package (mcp-server-fetch)cmd: uvx, args: [--with, mcp<2, …]thv run uvx://…fetch is also a docker image, so all three work here
Remote URL (already streamable HTTP)⚠️ only if runtimes available; else nothv run https://… (transparent proxy, no container)⚠️ only if HTTP-capablebest done with no bridgemcp.host straight to the URL
Already-HTTP server (in-cluster)n/an/an/askip the bridgemcp.host: http://server/mcp direct

The short rule: docker image or npm-only → ToolHive (or native stdio on a host with the runtimes); already-HTTP → mcp.host directly; teaching a bridge → mcpwrap.

Mechanism 1 — Context7-style per-user API keys (:18080)

The simplest mechanism, and the one Context7 popularized for MCP: hand each user a static secret key; the gateway maps the key to an identity via metadata. No browser popup, no token refresh — the key is the credential. agentgateway's apiKey gateway policy is documented under API key authentication.

Configuration

gateways:
  default:
    port: 8080
    # The default credential location is `Authorization: Bearer`; we pin it to
    # the x-api-key header (the sample's documented curl/test surface).
    apiKey:
      mode: strict
      location:
        header:
          name: x-api-key
      keys:
        # Shared automation/dev key (mcpuser)
        - key: sk-mcp-gateway-demo-key
          metadata:
            user: mcpuser
            role: admin
        # Per-user keys — one per developer (Context7-style)
        - key: sk-alice-demo-key
          metadata:
            user: alice
            role: dev
        - key: sk-bob-demo-key
          metadata:
            user: bob
            role: dev

Three things matter here:

  • mode: strict — requests without a valid key are rejected (401). The authentication mode docs also cover optional (validate if present) and permissive (never reject).
  • location.header.name: x-api-key — by default the key is read from Authorization: Bearer; pinning it to a custom header makes the curl/test surface unambiguous (credential location).
  • metadata.user — this is the identity. It becomes apiKey.user in CEL policies, so per-user quotas and attribution key off it.

Using it

# initialize (grab the Mcp-Session-Id response header)
curl -i -X POST http://localhost:18080/memory \
  -H "x-api-key: sk-alice-demo-key" \
  -H "content-type: application/json" \
  -H "accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'

MCP over HTTP is session-based: initialize returns a Mcp-Session-Id header, and subsequent tools/list / tools/call requests must echo it (see the HTTP transport section of the spec).

Per-user quotas with apiKey.user

Because the key maps to a user, rate limits can be conditional per user. agentgateway's local rate limiting supports CEL conditions:

localRateLimit:
  conditional:
    - condition: 'apiKey.user == "alice"'
      maxTokens: 60
      tokensPerFill: 60
      fillInterval: 1m
    - condition: 'apiKey.user == "bob"'
      maxTokens: 30
      tokensPerFill: 30
      fillInterval: 1m

Live verification: 40 rapid tools/list requests as bob (30/min limit) → 29×200 + 11×429 with error: rate limit exceeded, reason: RateLimit in the gateway logs. The per-user bucket works.

Revocation

Revocation is trivial: delete the key from the keys list (or rotate it). The config hot-reloads — no session to kill, no token to expire.

Best for: scripts, CI, MCP clients that can hold a secret. Not great for: humans in a browser (they'd have to copy keys around).

Mechanism 2 — mcpAuthentication: OAuth for MCP (Keycloak, :8082)

This is the mechanism the MCP ecosystem is converging on: OAuth 2.0 bearer tokens validated at the gateway, implementing the MCP Authorization specification. agentgateway's route-level mcpAuthentication policy is documented under MCP authentication.

What the policy does

Per the docs, agentgateway acts as an OAuth 2.0 resource server for MCP traffic, and with the keycloak provider it runs in the Authorization Server Proxy scenario:

  • Exposes protected-resource metadata on behalf of the MCP server (/.well-known/oauth-protected-resource/<path>).
  • Proxies Keycloak's authorization-server metadata and dynamic client registration (/.well-known/oauth-authorization-server/<path>).
  • Validates bearer tokens with Keycloak's live JWKS and returns 401 + WWW-Authenticate for unauthenticated requests.

The model is connect-time: the OAuth flow happens once when the client first connects; the access token is then reused for the whole session.

Configuration (one SSO route)

- name: memory-mcp-sso
  gateways: [sso]
  matches:
    - path: { pathPrefix: /memory } # the MCP endpoint
    - path: { exact: /.well-known/oauth-protected-resource/memory } # resource metadata
    - path: { exact: /.well-known/oauth-authorization-server/memory } # AS metadata (proxied)
    - path: { exact: /.well-known/oauth-authorization-server/memory/client-registration } # DCR
  policies:
    cors:
      allowOrigins: ['*']
      allowHeaders: ['*']
      exposeHeaders: ['Mcp-Session-Id']
    mcpAuthentication:
      mode: strict
      issuer: http://keycloak:8080/realms/mcp-demo
      audiences: [mcp-gateway]
      clientId: mcp-gateway
      jwks:
        url: http://keycloak:8080/realms/mcp-demo/protocol/openid-connect/certs
      provider:
        keycloak: {}
      resourceMetadata:
        resource: http://localhost:8082/memory
        scopesSupported: [read:all]
        bearerMethodsSupported: [header]

Field-by-field (all documented on the mcp-authn page):

FieldValue hereWhy
modestrictNo valid token → 401. optional/permissive exist for staging.
issuerhttp://keycloak:8080/realms/mcp-demoKeycloak realm URL; must match the token's iss and be resolvable from the gateway container (in-network keycloak:8080)
audiences[mcp-gateway]Required aud claim — the Keycloak client
jwks.urlKeycloak's live certs endpointNo static JWKS file — keys rotate transparently
provider.keycloak{}Adapts the proxy to Keycloak's non-standard endpoints (Authorization Server Proxy)
clientIdmcp-gatewayDCR short-circuit — see below
resourceMetadataresource/scope/bearer methodsWhat the gateway advertises in protected-resource metadata

The DCR puzzle (and the fix)

Dynamic Client Registration (RFC 7591) lets an MCP client like VS Code Copilot register itself on the fly. Out of the box, the gateway proxies client-registration to Keycloak's DCR endpoint — which returned 500 because Keycloak's DCR requires an initial access token.

The fix is documented in agentgateway's own docs: setting clientId on mcpAuthentication makes the gateway short-circuit registration with a mock 201 response — client_id: mcp-gateway, token_endpoint_auth_method: none, PKCE-only public client. No Keycloak initial-access token needed:

curl -s -X POST http://localhost:8082/.well-known/oauth-authorization-server/memory/client-registration \
  -H "content-type: application/json" \
  -d '{"redirect_uris":["http://127.0.0.1:33418"],"grant_types":["authorization_code"]}'
# → 201 {"client_id":"mcp-gateway","token_endpoint_auth_method":"none",...}

Why the routes must match the well-known paths

The policy serves the discovery metadata through the route — so every SSO route lists the four matches above. Without them, the gateway has no route handling /.well-known/oauth-* and the client can't bootstrap. Verified: both discovery endpoints return 200 without any token.

Keycloak side (realm export)

{
  "clientId": "mcp-gateway",
  "protocol": "openid-connect",
  "publicClient": true,
  "standardFlowEnabled": true,
  "directAccessGrantsEnabled": true,
  "redirectUris": [
    "http://localhost:3000/*",
    "http://127.0.0.1:33418",
    "http://127.0.0.1:33418/",
    "https://vscode.dev/redirect"
  ]
}

Two hard-won details:

  • publicClient: true — the client must be public (PKCE-only) for the DCR mock flow to work end-to-end. No client_secret anywhere.
  • Both 33418 variants — VS Code's callback is http://127.0.0.1:33418/ (trailing slash!). Keycloak matches redirect URIs exactly, so a missing trailing slash produced Invalid parameter: redirect_uri. Add both.
  • A read:all client scope must exist and be a default scope on the client, or Keycloak rejects the whole login with Invalid scopes: read:all (the gateway advertises scopesSupported: [read:all], so MCP clients request it). Persist it as a realm clientScopes entry with include.in.token.scope: true.

Real-world gotcha: jwt.sub is a UUID, not a username

Keycloak's sub claim is a random UUID (67c405a7-021d-47e3-b67f-aada53666876), not the login name. The login name lives in preferred_username. Every per-user rule in this sample keys off jwt.preferred_username — the single most important lesson from the whole build. Decode any token (jwt.io or the Keycloak admin console) before writing claim rules.

Mechanism 3 — mcpAuthorization: per-tool CEL rules

Authentication tells you who is calling; authorization tells you what they may do. agentgateway's mcpAuthorization policy runs against MCP method invocations (tools/list, tools/call, prompts, resources) rather than HTTP requests — documented under MCP authorization.

The killer feature: denied tools vanish

If a tool or other resource is not allowed, the gateway automatically filters it from the list response.

The client never even sees a tool it isn't allowed to call. That's a much better UX than a 403 on every call.

Rules with jwt.preferred_username

mcpAuthorization:
  rules:
    - 'jwt.preferred_username == "alice"' # rule 1
    - 'jwt.preferred_username == "bob"' # rule 2 (rules are OR-ed)
# tool-level rules with target backend (federated /mcp route)
mcpAuthorization:
  rules:
    - 'jwt.preferred_username == "alice"'
    - 'jwt.preferred_username == "bob" && mcp.tool.target != "fetch"' # bob: everything but fetch

Request-time CEL variables

Per the mcp-authz docs:

VariableMeaning
jwt.<claim>Claims from the token validated by mcpAuthentication — no separate auth policy needed
mcp.tool.nameThe tool being called (e.g. fetch)
mcp.tool.targetThe backend target handling the call (e.g. memory)
mcp.prompt.name / mcp.resource.namePrompt / resource access

⚠️ mcp.tool.arguments is not available during authorization — it is populated only after the tool call completes (access-log policies only). Base decisions on mcp.tool.name / mcp.tool.target.

Live-verified matrix

User/memory/fetch/thinking/mcp (federated)
alice9 toolsfetchsequentialthinking11 tools (all)
bob9 tools[] (denied)sequentialthinking10 tools (no fetch_fetch)

bob's tools/list on /fetch returns an empty list — the fetch tool was filtered, not erroring. This is the spec-compliant way to do least-privilege tool exposure.

Mechanism 4 — oidc: browser login for the Admin UI and web routes

The OIDC browser authentication policy is the gateway's human-facing login. It implements the OAuth 2.0 Authorization Code flow with PKCE and manages the session itself, so no separate oauth2-proxy is needed:

  1. An unauthenticated browser request hits a protected route.
  2. The gateway 302-redirects to the IdP login page (with a PKCE code_challenge).
  3. The user logs in at the IdP; the IdP redirects back with an authorization code.
  4. The gateway exchanges the code (with the code_verifier), validates the ID token against the IdP's JWKS, and sets an encrypted, tamper-proof session cookie carrying the ID-token claims.

Subsequent requests ride the cookie — the browser never handles raw tokens. PKCE is automatic, and the gateway always requests the openid scope. The session-cookie encryption is enabled by the OIDC_COOKIE_SECRET env var.

MCP clients are not browsers, so this policy is not for MCP traffic — it is for anything a human opens in a browser: the agentgateway Admin UI, a proxied dashboard, or any web route the gateway fronts.

Securing the agentgateway Admin UI

The Admin UI listens on ADMIN_ADDR (:15000 in this sample). Three ways to keep it private, from simplest to most integrated:

  1. Bind it to localhost onlyADMIN_ADDR=127.0.0.1:15000 in the compose file. Zero config, zero network exposure; reachable from the host (or via SSH port-forward) only.
  2. Gate it with an oidc-protected route — front the Admin port with a browser-facing route carrying the oidc policy below, so every browser hits the Keycloak login first.
  3. Reverse proxy in front — nginx/caddy (or a mesh sidecar) terminates OIDC and forwards to :15000; the pattern when the Admin listener itself cannot carry the policy.

Protecting a web route (sample)

The oidc policy attaches to a route (or a whole listener). Unlike the public mcp-gateway client used by mcpAuthentication, it needs a confidential Keycloak client (client secret), plus the cookie-secret env var:

export OIDC_COOKIE_SECRET="$(python3 -c 'import os; print(os.urandom(32).hex())')"
# Protect a browser-facing dashboard route with Keycloak login.
gateways:
  web:
    port: 8083 # browser-facing entry (published by the gateway)

routes:
  - name: dashboard-sso
    gateways: [web]
    matches:
      - path: { pathPrefix: / } # the dashboard (or anything the gateway fronts)
    policies:
      oidc:
        issuer: http://keycloak:8080/realms/mcp-demo
        clientId: agentgateway-browser
        clientSecret: change-me # confidential client in Keycloak
        redirectURI: http://localhost:8083/oauth/callback
        scopes: [profile, email] # `openid` is always added
    backends:
      - host: host.docker.internal:3000 # the dashboard app

Field-by-field (all documented on the oidc page):

FieldRequiredNotes
issueryesIdP issuer URL; discovery + JWKS are resolved from it
clientIdyesthe OAuth2 client registered in Keycloak
clientSecretyestoken-exchange credential — a confidential client (unlike the public mcp-gateway)
redirectURIyesgateway callback, e.g. http://localhost:8083/oauth/callback
scopesnoextra OAuth2 scopes to request; openid is always included
discovery / jwksnooverride the discovery document / JWKS location

Keycloak side: create a second client (e.g. agentgateway-browser) as a confidential client with standardFlowEnabled: true, directAccessGrantsEnabled: false, and redirect URI http://localhost:8083/oauth/callback; copy its secret into the policy. (The realm export in deployments/keycloak/realm-export.json shows the shape.) Access logs can then be enriched with the session identity via frontendPolicies.accessLog.add (e.g. user.id: jwt.sub, user.email: jwt.email).

Contrast with mcpAuthentication. Both hit the same Keycloak, but mcpAuthentication validates bearer JWTs and speaks MCP OAuth discovery/DCR for agents and IDEs; oidc manages an encrypted cookie session for human browsers. Same IdP, two client models — which is why the sample keeps them on separate gateways/ports. Mechanisms 1–3 are the live-verified set; the oidc route above follows the official docs and is the config to drop in when you front the Admin UI or a web route.

Bonus: the browser flow (VS Code Copilot)

Putting it all together, VS Code Copilot connects through the SSO port with a native oauth block in mcp.json — no manual token minting:

{
  "servers": {
    "memory": {
      "type": "http",
      "url": "http://localhost:8082/memory",
      "oauth": { "clientId": "mcp-gateway" },
    },
    "fetch": {
      "type": "http",
      "url": "http://localhost:8082/fetch",
      "oauth": { "clientId": "mcp-gateway" },
    },
    "thinking": {
      "type": "http",
      "url": "http://localhost:8082/thinking",
      "oauth": { "clientId": "mcp-gateway" },
    },
  },
}

The bootstrap sequence, all verified live:

  1. Copilot reads /.well-known/oauth-protected-resource/memory and /.well-known/oauth-authorization-server/memory (public, 200).
  2. It calls the advertised registration_endpoint → gateway answers with the mock 201 (PKCE-only public client).
  3. A browser popup opens at Keycloak's authorization endpoint (code_challenge=S256); the user logs in (alice/bob).
  4. Copilot exchanges the code (with code_verifier) at Keycloak's token endpoint and stores access + refresh tokens, auto-refreshing silently.

Each step maps to the manual PKCE commands in Where the playground token comes from — the code and code_verifier Copilot uses are the same ones you can mint by hand for the Tool Playground.

Getting the token from Keycloak for the agentgateway UI

If you are driving the agentgateway Admin UI Tool Playground manually, you need a bearer token. The fastest way is a password grant against the mcp-gateway public client. Keycloak is reachable on the host at http://localhost:8080 (the realm frontendUrl is http://keycloak:8080, so add 127.0.0.1 keycloak to your hosts file for browser flows):

curl -s -X POST http://localhost:8080/realms/mcp-demo/protocol/openid-connect/token \
  -d grant_type=password \
  -d client_id=mcp-gateway \
  -d username=alice \
  -d password=alice123

Keycloak returns a standard OAuth 2.0 token response (live output):

{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJ1d2YtV3FtS3QxVkg1LUpGeFBMZUIySkdQRWhHem9Tb09Zckl3QUlTZ0cwIn0...",
  "expires_in": 28800,
  "refresh_expires_in": 28800,
  "refresh_token": "eyJhbGciOiJIUzUxMiIsInR5cCIgOiAiSldUIiwia2lkIjog...",
  "token_type": "Bearer",
  "not-before-policy": 0,
  "session_state": "d4900961-937f-403c-9f65-ef1cb8f51245",
  "scope": "read:all"
}

The access_token is what you paste into the playground. Decoding it (jwt.io or jq -R 'split(".") | .[1] | @base64d | fromjson') shows the claims the gateway uses for identity and authorization:

ClaimValueUsed by
isshttp://keycloak:8080/realms/mcp-demomcpAuthentication issuer validation
audmcp-gatewaymcpAuthentication audience validation
preferred_usernamealicemcpAuthorization CEL rules, standardAttributes.user
scoperead:allKeycloak client default scope

⚠️ Remember: sub is a UUID, not the username. Rules use jwt.preferred_username.

Keycloak admin console view of the mcp-demo realm and the mcp-gateway public PKCE client:

Keycloak Administration Console — mcp-demo realm selected

Keycloak Administration Console — clients list showing mcp-gateway

Keycloak Administration Console — mcp-gateway client settings and redirect URIs

The sample users (alice, bob, mcpuser) are pre-created in the realm export:

Keycloak Administration Console — mcp-demo users list

Pasting the token into the agentgateway UI playground

With the token copied, open http://localhost:15000/ui/mcp/playground, expand Authorization header, paste the JWT into Bearer token, and click Initialize:

agentgateway Tool Playground — bearer token pasted and session initialized

After Initialize succeeds the tool list appears. In ToolHive/stdio mode that is 24 tools across the four federated targets:

agentgateway Tool Playground — federated tool dropdown with everything_echo visible

Select everything_echo, type a message, and click Call tool:

agentgateway Tool Playground — everything_echo response

The live response Echo: SSO verified through the UI! confirms the token made it through mcpAuthenticationmcpAuthorization → the federated backend and back.

Getting the token in VS Code Copilot

For VS Code Copilot you do not paste a token. Instead you add an oauth block to .vscode/mcp.json (or your user mcp.json) pointing at the SSO port:

{
  "servers": {
    "memory": {
      "type": "http",
      "url": "http://localhost:8082/memory",
      "oauth": { "clientId": "mcp-gateway" },
    },
    "fetch": {
      "type": "http",
      "url": "http://localhost:8082/fetch",
      "oauth": { "clientId": "mcp-gateway" },
    },
    "thinking": {
      "type": "http",
      "url": "http://localhost:8082/thinking",
      "oauth": { "clientId": "mcp-gateway" },
    },
  },
}

The oauth.clientId must match the clientId configured in mcpAuthentication (mcp-gateway in this sample). VS Code reads the protected-resource metadata, calls the DCR endpoint, then opens the Keycloak browser popup for PKCE login:

VS Code mcp.json configuration with oauth clientId for agentgateway

After you log in as alice, VS Code stores the access and refresh tokens in its secret storage and silently refreshes them. The agent side then sees the same namespaced tool list and the same per-user CEL filtering as the UI playground — bob will not see fetch_fetch if the mcpAuthorization rule is active.

One prerequisite bit us repeatedly: Keycloak's realm frontendUrl is http://keycloak:8080, so the browser must resolve that hostname. The fix is a hosts entry (127.0.0.1 keycloak) and Keycloak owning host port 8080 — which is why the gateway's apiKey port lives on 18080 in the compose file:

gateway:
  ports:
    - '18080:8080' # apiKey entry (host 8080 is reserved for Keycloak)
    - '8082:8082' # SSO entry

keycloak:
  ports:
    - '8080:8080' # Keycloak UI/issuer — the browser flow hits http://keycloak:8080
    - '8081:8080' # alternate host port for the admin console / tests

Testing each approach through the Admin UI

agentgateway ships a browser-based Admin UI on http://localhost:15000 (the :15000 port mapping in the compose files: "15000:15000" # agentgateway Admin UI). It is the fastest way to see the three approaches side by side, because the UI surfaces exactly what differs: how the gateway reaches each backend and how many tools come back. Start one approach at a time (./scripts/start-{stdio,toolhive,mcpwrap}.sh), open the UI, and run the two checks below.

Check 1 — MCP → Servers: how the gateway connects

The Servers page (http://localhost:15000/ui/mcp/servers) lists every configured backend and its connection type. This alone tells you which approach is running:

ApproachServers page showsWhy
1 — native stdio (host binary)4 servers, type Command Line (npx/uvx)the gateway spawns each server itself and owns the subprocesses
2 — ToolHive (production default)4 servers, type Streamable HTTPthe gateway talks HTTP to thv proxies on host.docker.internal:19001-19004
3 — mcpwrap (teaching only)3 servers, type Streamable HTTPmcpwrap proxies on :19101-19103; no everything (npm-only image)

All listed servers show as ready once the corresponding start script has finished. The everything server is the giveaway between the runtimes: it is npm-only (no docker image), so Approaches 1 and 2 host it (via npx / npx://) and show 4 servers, while Approach 3 — which hosts docker images only — stops at 3.

Check 2 — MCP → Tool Playground: initialize, authenticate, call a tool

The Tool Playground (http://localhost:15000/ui/mcp/playground) drives the real MCP listener: initialize a session, list the federated tools, and call one through the gateway. It uses the SSO port (:8082), so the auth step is exactly what a real MCP client would do against the Keycloak-protected endpoint:

# 1. Mint a Keycloak access token (realm mcp-demo, client mcp-gateway).
#    Tokens expire after 5 minutes — mint fresh right before Initialize.
curl -s -X POST http://localhost:8080/realms/mcp-demo/protocol/openid-connect/token \
  -d grant_type=password -d client_id=mcp-gateway \
  -d username=alice -d password=alice123

Then in the UI:

  1. Expand Authorization headerBearer token and paste the JWT from the access_token field of the response above.
  2. Click Initialize. Expect HTTP 200 and a session id that pins the active backends.
  3. Read the discovered tool count. This is the second runtime tell:
    • Approaches 1 and 2: 24 toolsmemory_* (9), fetch_fetch (1), thinking_sequentialthinking (1), everything_* (~13, including everything_echo).
    • Approach 3: 11 tools — the same set minus every everything_* tool.
  4. Pick a namespaced tool in the Tool combobox, fill its input schema, and click Call tool. The response proves the whole chain — UI session → gateway → runtime → spawned server — end to end:
    • everything_echo (Approaches 1/2) returns the text you sent, e.g. Echo: SSO verified through the UI.
    • thinking_sequentialthinking (any approach) returns a thought object, e.g. { "thoughtNumber": 1, "totalThoughts": 1, "nextThoughtNeeded": true, ... }.

The one gotcha: Keycloak access tokens live ~5 minutes. A stale token makes Initialize fail with 401; re-mint and re-paste. This is also why the get-mcp-token.sh script exists in samples/ — it wraps the curl above.

Where the playground token comes from: the PKCE flow

The password-grant curl above is the script-friendly shortcut. But the credential a modern MCP client really presents is minted by the PKCE flow — the exact sequence VS Code Copilot runs in the Bonus section. The gateway cannot tell the two apart: it validates every bearer token the same way (signature, issuer, audience, JWKS), so any valid access_token from client mcp-gateway works in the playground, however it was minted. PKCE is the part worth understanding, because the MCP authorization spec requires it for public clients and it makes a stolen authorization code useless:

  1. Generate a verifier and its S256 challenge.

    VERIFIER=$(openssl rand -base64 32 | tr '+/' '-_' | tr -d '=\n')
    CHALLENGE=$(printf '%s' "$VERIFIER" | openssl dgst -sha256 -binary | base64 | tr '+/' '-_' | tr -d '=\n')
    echo "verifier:  $VERIFIER"
    echo "challenge: $CHALLENGE"
    
  2. Open the authorization URL in a browser and log in as alice.

    http://localhost:8080/realms/mcp-demo/protocol/openid-connect/auth?client_id=mcp-gateway&response_type=code&scope=read:all&redirect_uri=http://127.0.0.1:33418/&code_challenge=CHALLENGE&code_challenge_method=S256&state=test
    

    Keycloak redirects to http://127.0.0.1:33418/?state=test&code=<CODE>. No listener needs to be running — the code is right in the address bar (this is exactly the redirect_uri the sample registers for Copilot, trailing slash included).

  3. Exchange the code, proving possession of the verifier.

    curl -s -X POST http://localhost:8080/realms/mcp-demo/protocol/openid-connect/token \
      -d grant_type=authorization_code -d client_id=mcp-gateway \
      -d code=CODE -d redirect_uri=http://127.0.0.1:33418/ \
      -d code_verifier=VERIFIER
    # -> access_token (~5 min) + refresh_token
    

    Paste the access_token into the playground's Bearer field and Initialize. Because PKCE binds the code to the verifier, an attacker who intercepts the redirect can never redeem it without the verifier.

StepPlayground (manual)VS Code CopilotArtifact
Metadata discoverycurl the /.well-known endpoints (optional)automaticAS metadata
Client registrationnot needed — mcp-gateway already existsmock 201 (the clientId DCR short-circuit)client_id
Authorizebrowser URL + alice loginpopup + alice logincode
Token exchangecurl with code_verifierinternal exchange, silent refreshaccess_token + refresh_token

In VS Code specifically: the oauth block in mcp.json runs these steps with its own verifier — discovery, DCR, popup, exchange, and auto-refresh when the token nears expiry (the token lives in VS Code's secret storage, so you never see it in a terminal). To inspect the token it actually presented: catch the redirect code in the login popup's address bar (or its network tab), then exchange it yourself with the commands above; or paste any minted access_token into jwt.io and confirm iss, aud, and preferred_username. For the playground, minting a fresh token with either flow gives an equivalent credential.

The same OAuth flow in Claude Code and Copilot CLI

The MCP authorization spec is implemented by the other terminal agents too, so the same :8082 SSO gateway, same Keycloak, same PKCE flow — no extra server work. Both read the Claude-style .mcp.json (mcpServers wrapper) and both auto-refresh the stored token:

Claude Code — add the server, then trigger the flow from the shell:

claude mcp add --transport http memory http://localhost:8082/memory
claude mcp login memory    # or /mcp inside a session

Claude Code discovers the OAuth metadata exactly the way agentgateway serves it — RFC 9728 protected-resource metadata first, then RFC 8414 authorization-server metadata — so mcpAuthentication's /.well-known routes work as-is and the clientId short-circuit answers DCR with the mock 201. An explicit oauth block pins the client or fixes the callback port:

{
  "mcpServers": {
    "memory": {
      "type": "http",
      "url": "http://localhost:8082/memory",
      "oauth": { "clientId": "mcp-gateway", "callbackPort": 33418 },
    },
  },
}

Copilot CLI — same .mcp.json, managed via copilot mcp:

copilot mcp add ...    # or edit .mcp.json; /mcp auth re-authenticates

Copilot CLI performs MCP OAuth 2.0 with automatic token management and refresh, and falls back to the RFC 8628 device-code flow in headless or CI terminals — no browser redirect needed. Its config keys are oauthClientId and auth.redirectPort (legacy oauth.clientId / oauth.callbackPort are auto-migrated). For one-shot sessions: copilot --additional-mcp-config '{"mcpServers": { ... }}'.

ClientAdd / configOAuth triggerToken handlingHeadless / CI fallback
VS Code Copilotmcp.json oauth blockpopup on first connectsecret storage, silent refresh
Claude Codeclaude mcp add --transport http/mcp or claude mcp loginsystem keychain, auto-refreshprints auth URL, paste redirect
Copilot CLIcopilot mcp add / .mcp.json/mcp authsecure store, auto-refreshdevice-code flow (RFC 8628)

One Keycloak gotcha for all three: the token exchange redirects to Keycloak, which matches redirect_uri exactly (lesson 4 below). VS Code's http://127.0.0.1:33418/ is registered in the sample; Claude Code and Copilot CLI pick a random local callback port by default — either fix the port (--callback-port / auth.redirectPort) and register that exact URI in the mcp-gateway client, or add a wildcard such as http://localhost:33418/* to its redirectUris.

What Keycloak auth proves in the UI

The playground authenticates as the pasted user, and the gateway enforces the full mcpAuthentication + mcpAuthorization policy on every request — not just in the UI, but on any client that connects to :8082/mcp:

IdentityToken sourceResult in the playground
aliceusername=alice password flowall 24 tools visible; everything_echo and thinking_sequentialthinking callable
bobusername=bob password flow23 tools — fetch_fetch is absent (filtered by the CEL rule, not erroring)

The per-user filtering comes from the mcpAuthorization rule (jwt.preferred_username == "bob" && mcp.tool.target != "fetch" in the top-level mcp: block), so bob never even sees the fetch tool to attempt it. The API-key mechanism on :18080/mcp is the no-OAuth alternative: x-api-key: sk-alice-demo-key returns the same federated 24-tool view for curl/scripts.

The screenshots below were captured live from the Admin UI while each approach was running:

ScreenshotApproachWhat it shows
stdio-servers.png1 — native stdioMCP → Servers lists 4 targets as Command Line (npx/uvx)
stdio-playground-initialized.png1 — native stdioMCP → Tool Playground initialized with 24 tools
toolhive-servers.png2 — ToolHiveMCP → Servers lists 4 targets as Streamable HTTP
toolhive-playground-initialized.png2 — ToolHiveMCP → Tool Playground initialized with 24 tools
mcpwrap-servers.png3 — mcpwrapMCP → Servers lists 3 targets as Streamable HTTP
mcpwrap-playground-initialized.png3 — mcpwrapMCP → Tool Playground initialized with 11 tools

Verified live, one approach at a time — this table is what the two checks above produce in this sample:

ApproachServers pagePlayground initializeTool countSample call result
1 stdio4 × Command LineHTTP 20024everything_echo → "Echo: stdio approach verified"
2 ToolHive4 × Streamable HTTPHTTP 20024everything_echo → "Echo: SSO verified through the UI"
3 mcpwrap3 × Streamable HTTPHTTP 20011thinking_sequentialthinking → thought object

When you are done, tear each stack down with its matching stop-*.sh script (./scripts/stop-stdio.sh, stop-toolhive.sh, stop-mcpwrap.sh) — the scripts stop the host-side runtime (binary / thv workloads / mcpwrap daemon) and docker compose down the gateway + Keycloak + observability stack. (The vMCP variant has no Admin UI — it is verified with initialize + tools/list on :4483/mcp instead; see Approach 4 and its ./scripts/stop-vmcp.sh.)

Choosing between the mechanisms

API key (Context7-style)mcpAuthentication (Keycloak OAuth)oidc (browser session)
CredentialStatic per-user secret (x-api-key)Short-lived JWT + refresh (browser login)Encrypted session cookie (IdP login)
IdentityapiKey.user (key metadata)jwt.preferred_username (claims)IdP ID-token claims (jwt.*)
ExpiryNone until revokedAccess-token + session expiry, refresh flowCookie session (IdP-managed)
RevocationDelete key from configDisable user in KeycloakIdP session/account control
Browser UXNone — caller sends the keyPopup login, PKCE, silent refreshFull redirect login, cookie-based
Best forScripts, CI, secret-holding clientsHuman developers, Copilot oauth block, per-user scopesAdmin UI, web dashboards, browser routes

All three coexist in the same config (different gateways/ports) and all feed the same downstream CEL policies (localRateLimit, logs, traces) — the only difference is which identity claim the rules read (apiKey.user for keys, jwt.* for OAuth and OIDC sessions).

The four security mechanisms — full docs map

agentgateway's security docs index four mechanisms; this sample demonstrates all four — three on the MCP traffic itself, and the fourth (OIDC) on browser-facing routes (the Admin UI and web dashboards):

DocFeatureIn this sample?Used for
API key authenticationstatic per-user secret keys:18080scripts, CI, service-to-service, secret-holding clients
MCP authenticationOAuth 2.0 resource server (MCP auth spec):8082MCP clients — agents, IDEs, Copilot oauth block
MCP authorizationper-tool CEL rules:8082hide denied tools from the client's view entirely
OIDC browser authenticationIdP login + encrypted session cookie✅ config abovebrowser-facing routes — Admin UI, web dashboards

OIDC vs the MCP mechanisms. The oidc policy is a browser flow: unauthenticated requests are redirected to the IdP login, the gateway exchanges the code with PKCE, and the session lives in an encrypted cookie (OIDC_COOKIE_SECRET). MCP clients are not browsers — agents, IDEs, and scripts authenticate with bearer tokens per the MCP authorization spec, not cookies. So for MCP traffic the right pair stays mcpAuthentication (authn) + mcpAuthorization (authz), exactly what the sso gateway runs; reach for oidc when a human browser must log in — the Admin UI or any web route the gateway fronts (Mechanism 4 above).

Beyond the four documented mechanisms, the agentgateway security index and the MCP ecosystem offer more layers. Popularity is about what teams actually deploy today; security posture is about the credential model and blast radius:

Security layerMechanism (agentgateway)PopularitySecurity postureFits
AuthN — agents/IDEsmcpAuthentication (OAuth 2.1 + PKCE + DCR)ecosystem standardhigh — short-lived JWTs, refresh, revocationmodern MCP clients — the primary
AuthN — servicesjwtAuth (JWKS / issuer / audiences)commonhigh — same token validation, no MCP discoveryagents/services already holding a token (workload identity, CI OIDC)
AuthN — browseroidc (encrypted session cookie)commonhigh — PKCE enforced, tamper-proof cookieAdmin UI, dashboards, web routes
AuthN — simpleAPI keys (apiKey)most commonmedium — static, leak-pronescripts, CI, quick start
AuthN — legacybasic auth / client certslegacylow–mediumlegacy clients; mTLS only at the edge
AuthZ — per-toolmcpAuthorization (CEL rules)emerginghigh — denied tools vanishleast-privilege tool exposure
AuthZ — externalexternal authz (OPA)nichehigh — policy as codeorg-wide policy and audit
QuotalocalRateLimitcommonprotectivecost control, per-user buckets
NetworkToolHive egress/DNS isolation, networkAuthzemerginghigh — SSRF guarduntrusted servers
In transitbackend TLS / gateway TLSmandatoryhigheverything outside localhost

The most popular: API keys (ubiquity) and OAuth via mcpAuthentication — the latter is what the MCP spec prescribes and what Copilot/Claude/Cursor implement natively, so it is the default for modern agentic clients.

The most secure (defense in depth — this sample's full stack): mcpAuthentication (strict, Keycloak), mcpAuthorization (per-tool CEL), localRateLimit, per-user standardAttributes attribution, and ToolHive network isolation for untrusted servers (+ TLS on any real deployment). Static keys and cookies are the weak spots; short-lived OAuth JWTs with refresh are the strong core.

Which one is recommended for MCPs? mcpAuthentication (strict) + mcpAuthorization + per-user rate limits, backed by Keycloak — i.e. exactly what this sample's sso gateway already runs:

  • Why: it implements the MCP authorization spec — what real MCP clients (the 2025-03-26 protocol, IDE integrations, agents) natively expect: OAuth discovery (/.well-known/oauth-authorization-server), dynamic client registration, the resource param (RFC 8707), PKCE, and standard 401 + WWW-Authenticate challenges.
  • Identity > keys: real users, expiry, revocation, federated IdP. An API key can't express "alice logged in via corporate SSO".
  • Per-tool authz: CEL rules filter denied tools from the client's view entirely — bob never even sees fetch.
  • API keys: fine as a quick start / internal service-to-service fallback (simplest, zero infra), or alongside SSO for machine clients that can't do OAuth. That's why the sample keeps both on separate gateways — a good pattern.
  • Stacking (what this sample demonstrates end-to-end): Keycloak as IdP → mcpAuthentication (authn) → mcpAuthorization (authz) → localRateLimit (quota) → OTel traces with standardAttributes.user. All four mechanisms live-verified.

Observability: attribution across the stack

Every mechanism feeds per-user attribution into the observability stack (OTLP → otel-collector → Tempo/Prometheus/Loki → Grafana):

config:
  tracing:
    otlpEndpoint: http://otel-collector:4317
    otlpProtocol: grpc
    randomSampling: true
  logging:
    format: json
    level: info
    database:
      url: sqlite:///var/log/agentgateway/request-log.db
  standardAttributes:
    # identity for every request log/metric/trace:
    # JWT preferred_username on :8082, API-key metadata user on :18080
    user: 'has(jwt.preferred_username) ? jwt.preferred_username : apiKey.user'

Gateway stdout logs are pushed straight to Loki by the Docker loki log driver (attached to the gateway container in deployments/docker-compose.toolhive.yml — no collector filelog leg needed since the gateway is the stock image), while the collector fans out traces → Tempo, OTLP logs → Loki, metrics → Prometheus. Verified live: gateway request logs queryable in Loki (labels container_name / job="agentgateway"); traces carry jwt.* span attributes on the SSO port.

Lessons learned (the short version)

  1. jwt.sub is a UUID. Use preferred_username for per-user rules.
  2. mcpAuthentication populates jwt.* claims without any separate auth policy — route-level CEL (mcpAuthorization, localRateLimit) reads them directly. The old "claims need jwtAuth" belief was wrong.
  3. DCR 500 → set clientId on mcpAuthentication for the mock-201 short-circuit, and make the Keycloak client public (PKCE).
  4. Keycloak matches redirect URIs exactly — register both 33418 and 33418/.
  5. Register every advertised scope as a Keycloak default client scope, or login dies with Invalid scopes.
  6. SQLite request-log caveat — the current gateway image writes a WAL that never checkpoints (0 committed rows); attribution is best observed via stdout logs / traces / the Admin UI.
  7. PowerShell mangles inline JSON — write MCP bodies to a file and use curl --data-binary "@file".
  8. "legacy MCP backend" in Traffic → Routes is cosmetic — the admin UI hardcodes that label (ui/src/traffic.ts) for any route whose backend is mcp: (routing-based mode). It's not a deprecation: per the configuration-modes docs, routes[].backends[].mcp and the top-level mcp: section are two forms of the same MCP backend. The label only goes away if you use the top-level mcp: section (shown under MCP → Servers instead) — but that shares one endpoint and one policy set per gateway, so it can't express separate /memory /fetch /thinking paths with different auth per port. That's exactly why the sample keeps the routing-based form for the per-server routes and uses the top-level mcp: block only for the federated :8082/mcp the Tool Playground connects to.

Conclusion

Securing MCP servers is a layered problem. Use static API keys for machines, OAuth (PKCE + DCR) for people, and per-tool CEL authorization on top. Per-user attribution should follow each request. Put OIDC in front of the browser-facing control plane, such as the Admin UI, so people do not need to handle API keys. agentgateway's route-level model brings these pieces together: mcpAuthentication for authn, mcpAuthorization for authz, oidc for browser sessions, localRateLimit for quotas, and standardAttributes for observability. They can all read the same identity claims.

The full, runnable sample — with code for all four approaches, from Approach 1 through the optional vMCP variant — lives in samples/mcp-gateway-agentgateway/ in this repository — bring it up with scripts/start-toolhive.sh (tear down with scripts/stop-toolhive.sh; the recommended Approach 2 with the full four-server fleet), or run one of the other two variants — the native stdio host binary (scripts/start-stdio.sh / scripts/stop-stdio.sh, no custom image) or the teaching mcpwrap wrapper (scripts/start-mcpwrap.sh / scripts/stop-mcpwrap.sh) — one variant at a time, same gateway, same ports. And if you want to see the same multiplexing and Keycloak OAuth implemented by ToolHive without a gateway, run the optional vMCP variant (scripts/start-vmcp.sh / scripts/stop-vmcp.sh, Approach 4) — one endpoint on :4483/mcp, 24 prefixed tools, the same alice/bob tokens. Log in through VS Code Copilot with alice, and watch the tool lists differ between users — or drive the same checks from the Admin UI's Servers page and Tool Playground (see Testing each approach through the Admin UI). The gateway's own docs (authentication, authorization, API keys, and the OIDC browser flow for cookie-based sessions) are the best next step — and the MCP authorization spec explains why these shapes exist.

Start with API keys for automation. Move to Keycloak OAuth when people and IDE clients need access. Add mcpAuthorization rules when users need different tool sets; the gateway removes denied tools from their view.