Building an AI Gateway with AgentGateway: LLM, MCP, and A2A Behind One Door

Introduction

Any AI application that calls a model has to answer a few operational questions. Which model should handle a request? Who may call it? How do we keep a client from sending an unbounded prompt history? How do we rate-limit one API key instead of every caller together? Once agents start using tools, the gateway has to govern those calls as well as model traffic.

Many teams answer these questions in application code, one if statement at a time. That works for one application. By the time a second or third service arrives, each service may have its own version of the same authentication, guardrail, and metering logic.

This guide puts those concerns in one place with AgentGateway, the Linux Foundation's Rust gateway. The deployment fronts an LLM provider, five native MCP servers, one OpenAPI-generated MCP target, and an A2A agent. It also centralizes keys, authentication, authorization, rate limits, guardrails, and observability. The runnable sample is in the blog-samples/agentgateway-ai-gateway repository. I mark Compose features separately from patterns that need additional infrastructure.

The stack covers:

An LLM gateway that fronts DeepSeek with virtual API keys and a weighted virtual model.
An MCP gateway that multiplexes six targets behind one endpoint: five tool servers and an OpenAPI Petstore adapter. Every tool is prefixed with its target name.
Authentication for MCP traffic through Keycloak (OAuth2/OIDC with PKCE for browser clients) and authorization through CEL rules.
Guardrails that reject prompt-injection attempts and PII in model output.
Rate limiting with an in-memory default and an optional Envoy-compatible remote setup.
An A2A route that exposes a .NET agent through the gateway.
Observability with OpenTelemetry traces, Prometheus metrics, an LGTM stack, and self-hosted Langfuse for LLM traces.

MCP deep dive. For the MCP security and runtime details, read Running MCP Servers in Production: agentgateway + ToolHive, from stdio to a Multiplexed HTTP Endpoint. It focuses on Keycloak authentication, CEL authorization, stdio hosting, ToolHive, and vMCP.

Use case

Consider a support platform where users report tickets, browse a product catalog, and ask an AI assistant for help. The assistant reads tickets, searches the catalog, looks up customers, and answers questions about the time or echo tools used in demos. DeepSeek provides the LLM, while the tools come from a mix of .NET services and third-party reference servers.

Without a gateway, the assistant would hold the DeepSeek key, know every tool URL, repeat role checks for each tool, and produce its own metrics. With one in front, the assistant uses one address and requests the virtual model deepseek-smart; the gateway selects the concrete model. When the assistant lists tools, it receives only the tools that user may call. Each call passes through the gateway's rate limit and prompt-injection checks before it reaches a backend.

The sample uses a console client named SupportChat, several MCP tool servers, an A2A agent named SupportAgent, and the gateway between them.

Architecture overview

Rendering diagram...

flowchart LR
    subgraph Clients
        SC[SupportChat .NET console]
        B[Browser - MCP Tool Playground]
        UI[Admin UI :15000]
    end

    subgraph Gateway[AgentGateway]
        LG[LLM API-key gateway :4000]
        LO[LLM OIDC/PKCE gateway :4001]
        MG[MCP gateway :3000]
        AG[A2A route :3001]
        M[Prometheus metrics :15020]
    end

    subgraph LLM[Upstream LLM]
        DS[DeepSeek]
    end

    subgraph MCP[MCP targets]
        T[.NET services]
        EV[everything via ToolHive :19101]
        TM[time service :8084]
    end

    subgraph Observability
        OC[OpenTelemetry Collector]
        GF[Grafana :13000]
        LF[Langfuse :13001]
    end

    SC --> LG
    SC --> MG
    SC --> AG
    B --> LO
    UI --> M
    LG --> DS
    LO --> DS
    MG --> T
    MG --> EV
    MG --> TM
    AG --> SA[SupportAgent :9999]
    LG --> OC
    MG --> OC
    AG --> OC
    OC --> GF
    OC --> LF

AgentGateway overview showing the running gateway configuration

Prerequisites

Docker with the compose plugin.
ToolHive (thv) installed, to run the third-party everything MCP on the host.
A DeepSeek API key. The compose stack reads it from DEEPSEEK_API_KEY in deploy/.env.example.
The .NET SDK (9 or 10) if you want to run SupportChat from source instead of inside the stack.
curl, python3, and optionally jq for the verification script.

The gateway image is cr.agentgateway.dev/agentgateway:latest and runs with -f /config.yaml. Most supporting images use floating tags, while Loki is pinned to 3.3.2 so log ingestion behavior remains reproducible.

Docker deployment

The sample has three deployment layers: the gateway config, the Compose stack, and two shell scripts for the pieces Compose does not manage.

The Compose stack

deploy/docker-compose.yaml defines the full stack:

name: agentgateway-ai-gateway

services:
  agentgateway:
    image: cr.agentgateway.dev/agentgateway:latest
    command: ['-f', '/config.yaml']
    depends_on:
      keycloak:
        condition: service_healthy
      otel-collector:
        condition: service_started
      mcp-tickets:
        condition: service_started
      mcp-catalog:
        condition: service_started
      mcp-customers:
        condition: service_started
    read_only: true
    cap_drop: [ALL]
    security_opt:
      - no-new-privileges:true
    extra_hosts:
      - 'host.docker.internal:host-gateway'
    volumes:
      - ./agentgateway-config.yaml:/config.yaml:rw
      - gateway-logs:/var/log/agentgateway

The gateway container is hardened: read-only rootfs, all capabilities dropped, and no docker.sock mount. The SQLite request log uses a named volume. The config file is the one deliberate writable bind mount because the Admin UI persists changes such as Apply CORS; this does not make the container filesystem writable.

The stack also runs the three first-party MCP servers (mcp-tickets, mcp-catalog, mcp-customers), the A2A agent (support-agent), Keycloak, the OpenTelemetry collector, the LGTM services (Prometheus, Tempo, Loki, Grafana), and Langfuse with its Postgres and Redis.

Why ToolHive runs the external MCP

One of the five native MCP targets is a third-party reference server: everything (npx-only). It is stdio-only by default, which means the Docker gateway cannot reach it over HTTP on its own. ToolHive (thv) wraps it in a streamable HTTP proxy on the host. The sixth target is the OpenAPI Petstore adapter described later:

thv run npx://@modelcontextprotocol/server-everything@latest \
  --name mcp-everything --host 0.0.0.0 --proxy-port 19101 \
  --transport stdio --proxy-mode streamable-http --isolate-network=false

The gateway reaches it through host.docker.internal:19101, made resolvable by the extra_hosts entry in compose. The sample's time tool is a small first-party .NET MCP service at mcp-time:8084/mcp; it runs directly in Compose because it natively speaks streamable HTTP.

AgentGateway MCP server list showing multiplexed targets

The split is practical. First-party MCP servers speak streamable HTTP, so Compose manages and restarts them. The third-party stdio server runs through a ToolHive proxy on the host. Because that proxy stops when its terminal closes, scripts/start-mcps.sh starts it on each run.

./scripts/start-mcps.sh            # ToolHive MCPs + compose up --build
./scripts/start-mcps.sh --ratelimit  # adds the Envoy rate-limit override
./scripts/stop-mcps.sh             # inverse

After startup, SupportChat runs against everything from the host:

cd src/SupportChat
dotnet run

The client logs into Keycloak, lists the multiplexed tools, runs three chat turns that trigger MCP tool calls, and finally calls the A2A agent.

The LLM gateway

The llm section of the config attaches to the implied default gateway on port 4000. It declares the real DeepSeek models, a virtual model on top of them, virtual API keys, guardrails, and rate limits.

AgentGateway's LLM configuration guide explains when to use simplified model configuration versus routing-based configuration. This sample uses both: simplified llm configuration for the API-key endpoint and a routing-based route for the browser OIDC endpoint.

Models and virtual models

llm:
  models:
    - name: deepseek-chat
      visibility: public
      provider: deepseek
      params:
        apiKey: '$DEEPSEEK_API_KEY'
        model: deepseek-chat
    - name: deepseek-reasoner
      visibility: public
      provider: deepseek
      params:
        apiKey: '$DEEPSEEK_API_KEY'
        model: deepseek-reasoner

  virtualModels:
    - name: deepseek-smart
      routing:
        weighted:
          targets:
            - model: deepseek-chat
              weight: 70
            - model: deepseek-reasoner
              weight: 30

Clients request deepseek-smart. The gateway spreads 70 percent of traffic to deepseek-chat and 30 percent to deepseek-reasoner. The client never knows which concrete model answered, and you can rebalance or reroute without touching a single client. The provider is deepseek, whose default base URL is https://api.deepseek.com/v1; the real key lives only in the gateway.

AgentGateway model configuration with weighted virtual model targets

Prompt enrichment

The browser LLM route prepends a short system message before forwarding chat and Responses API requests. This keeps shared support-agent guidance at the gateway boundary while leaving service clients on the API-key endpoint free to choose their own prompts:

policies:
  ai:
    routes:
      /v1/chat/completions: completions
      /v1/responses: responses
    prompts:
      prepend:
        - role: system
          content: You are the support platform assistant. Prefer concise, actionable answers.

AgentGateway supports both prepend and append prompt lists. Keep injected content short, versioned, and visible to operators because it changes model behavior without changing client code. See the official prompt enrichment example.

The .NET client talks to the gateway as if it were an OpenAI-compatible endpoint:

var openAiOptions = new OpenAIClientOptions
{
    Transport = new HttpClientPipelineTransport(
        new HttpClient { BaseAddress = new Uri(GatewayLlmUrl + "/") }
    ),
};
var openAiClient = new OpenAIClient(new ApiKeyCredential(GatewayApiKey), openAiOptions);
var chatClient = openAiClient.GetChatClient("deepseek-smart").AsIChatClient();

Virtual API keys

A strict virtual key policy turns the gateway into the only place that knows real credentials:

llm:
  policies:
    apiKey:
      mode: strict
      keys:
        - key: sk-alice-abc123def456
          metadata:
            user: alice
        - key: sk-bob-xyz789uvw012
          metadata:
            user: bob

mode: strict means every request must carry one of these keys. The metadata.user value becomes apiKey.user in CEL expressions, and it flows into metrics and logs:

config:
  metrics:
    fields:
      add:
        user_id: apiKey.user

This field lets Grafana group token usage by Alice, and lets the rate limiter target a person instead of an IP address.

API keys and browser OAuth use separate entry points

The sample deliberately does not make one LLM request satisfy two unrelated authentication systems. The service-client endpoint on :4000 uses strict virtual API keys. The browser endpoint on :4001 uses AgentGateway's oidc policy, which implements the Authorization Code flow with PKCE and stores the authenticated browser session in an encrypted cookie.

That split is useful in practice. A backend worker can send a short-lived or rotated API key without pretending to be a browser. A user-facing web client can redirect a person to Keycloak without putting an API key in JavaScript. The two endpoints can still point at the same provider and model catalog.

The browser route is a normal routing-based AgentGateway route:

gateways:
  llm-browser:
    port: 4001

routes:
  - name: llm-browser-oidc
    gateways: [llm-browser]
    matches:
      - path:
          pathPrefix: /v1
      - path:
          exact: /oauth/callback
    policies:
      oidc:
        issuer: http://localhost:8080/realms/agentgateway
        clientId: agentgateway-browser
        clientSecret: agentgateway-browser-secret
        redirectURI: http://localhost:4001/oauth/callback
        authorizationEndpoint: http://localhost:8080/realms/agentgateway/protocol/openid-connect/auth
        tokenEndpoint: http://keycloak:8080/realms/agentgateway/protocol/openid-connect/token
        jwks:
          url: http://keycloak:8080/realms/agentgateway/protocol/openid-connect/certs
        scopes: [profile, email]
    backends:
      - ai:
          name: deepseek-browser
          provider:
            openAI:
              model: deepseek-chat
          hostOverride: api.deepseek.com:443
          policies:
            backendAuth:
              key: '$DEEPSEEK_API_KEY'

The callback path matters. The OIDC middleware receives the authorization code on /oauth/callback, exchanges it with Keycloak using the PKCE verifier, sets the session cookie, and redirects back to the original /v1 request. If the route matches only /v1, login succeeds at Keycloak but the callback returns 404 at the gateway.

AgentGateway generates the PKCE values. A request to http://localhost:4001/v1/models produces a redirect containing code_challenge and code_challenge_method=S256:

GET /v1/models
302 Location: http://localhost:8080/realms/agentgateway/protocol/openid-connect/auth
    ?client_id=agentgateway-browser
    &response_type=code
    &code_challenge=...
    &code_challenge_method=S256

See the official OIDC browser authentication guide, especially its session-management and PKCE sections.

The MCP gateway: six targets, one endpoint

The mcp section attaches to the implied default gateway on port 3000. Its targets list is where multiplexing happens:

mcp:
  targets:
    - name: tickets
      mcp:
        host: http://mcp-tickets:8081/mcp
    - name: catalog
      mcp:
        host: http://mcp-catalog:8082/mcp
    - name: customers
      mcp:
        host: http://mcp-customers:8083/mcp
    - name: everything
      mcp:
        host: http://host.docker.internal:19101/mcp
    - name: time
      mcp:
        host: http://mcp-time:8084/mcp

One endpoint, six targets. Every tool is prefixed with its target name, so the everything server's echo tool appears as everything_echo and the tickets server's list tool as tickets_tickets_list. Name collisions between targets are impossible by construction, and the client sees one flat tool list. This is AgentGateway MCP multiplexing: the client opens one session, while the gateway owns the target connections and fans requests out to the correct server.

A raw tools/list request looks like this:

curl http://localhost:3000/mcp \
  -X POST \
  -H "Authorization: Bearer $KEYCLOAK_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'

The response is one aggregated list. In this sample it contains names from all six targets, including values such as:

{
  "result": {
    "tools": [
      { "name": "tickets_tickets_list" },
      { "name": "catalog_catalog_search" },
      { "name": "customers_customers_get" },
      { "name": "everything_echo" },
      { "name": "time_get_current_time" }
    ]
  }
}

Calling a tool uses the prefixed name. The client does not select a backend URL or know whether that backend is a container or a ToolHive proxy:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "catalog_catalog_search",
    "arguments": { "query": "laptop" }
  }
}

AgentGateway's MCP backend documentation describes HTTP, stdio, and session-routing behavior. The sample uses direct HTTP for its .NET servers and ToolHive only where a third-party server starts as stdio.

The first-party servers are small ASP.NET Core apps using the .NET MCP SDK. Each one registers its tools as a class and maps the endpoint:

builder.Services.AddMcpServer()
    .WithHttpTransport(options =>
    {
        options.SessionMode = HttpServerSessionMode.Stateless;
    })
    .WithTools<TicketTools>();

var app = builder.Build();
app.MapMcp("/mcp");
app.Run();

Stateless mode is the recommended setting for the 2026-07-28 streamable HTTP wire format: no Mcp-Session-Id, no in-memory session state, and horizontal scaling without session affinity. The gateway can round-robin tool calls across replicas freely.

The tool class uses [McpServerTool] attributes and XML docs for descriptions:

[McpServerToolType]
public sealed class TicketTools
{
    /// <summary>
    /// List support tickets, optionally filtered by status.
    /// </summary>
    [McpServerTool(Name = "tickets_list")]
    public IReadOnlyList<Ticket> ListTickets(string? status = null)
    {
        return string.IsNullOrWhiteSpace(status)
            ? Tickets
            : Tickets.Where(t => t.Status.Equals(status, StringComparison.OrdinalIgnoreCase)).ToList();
    }
}

The MCP client on the other side is equally small. SupportChat connects through the gateway with a Keycloak bearer token and gets back the full federated tool list:

var transport = new HttpClientTransport(
    new HttpClientTransportOptions
    {
        Endpoint = new Uri(GatewayMcpUrl),
        AdditionalHeaders = new Dictionary<string, string>
        {
            ["Authorization"] = $"Bearer {accessToken}",
        },
    },
    loggerFactory
);

await using var mcpClient = await McpClient.CreateAsync(
    transport,
    new McpClientOptions
    {
        ClientInfo = new Implementation { Name = "support-chat", Version = "1.0.0" },
    },
    loggerFactory
);

var tools = await mcpClient.ListToolsAsync();

Those tools then become callable from the chat pipeline via Microsoft.Extensions.AI function invocation:

chatClient = chatClient
    .AsBuilder()
    .UseFunctionInvocation(
        null,
        options =>
        {
            foreach (var tool in tools)
            {
                options!.AdditionalTools.Add(tool);
            }
        }
    )
    .Build();

Now a single chat turn can decide to call tickets_tickets_list or catalog_catalog_search, and every one of those calls crosses the gateway, gets authorized, gets rate-limited, and gets traced.

MCP authentication: Keycloak, OAuth, and PKCE

The MCP gateway requires a valid Keycloak access token. The mcpAuthentication policy makes AgentGateway an OAuth-protected MCP resource server. It validates JWT signatures, exposes protected-resource metadata, proxies Keycloak's OAuth metadata and client registration endpoints, and makes the resulting claims available to MCP authorization rules.

mcp:
  policies:
    mcpAuthentication:
      mode: strict
      issuer: http://localhost:8080/realms/agentgateway
      audiences:
        - agentgateway
      jwks:
        url: http://keycloak:8080/realms/agentgateway/protocol/openid-connect/certs
      provider:
        keycloak: {}
      clientId: agentgateway-browser
      resourceMetadata:
        resource: http://localhost:3000/mcp
        scopesSupported:
          - read:all
        bearerMethodsSupported:
          - header

The public URLs used by browser clients are exposed through the OAuth metadata endpoints:

GET /.well-known/oauth-protected-resource/mcp
GET /.well-known/oauth-authorization-server/mcp
POST /.well-known/oauth-authorization-server/mcp/client-registration

The authorization-server metadata advertises code_challenge_methods_supported including S256. MCP clients can therefore discover Keycloak and complete the Authorization Code + PKCE flow without hard-coding Keycloak's token endpoint. The clientId setting makes registration deterministic for this sample; the gateway returns the pre-registered browser client instead of creating a new client on every run.

Keycloak is provisioned with a realm called agentgateway, two roles (support-admin, support-user), and two users: Alice (admin) and Bob (user). It also defines two clients that matter for the two MCP consumers in this sample:

agentgateway-browser, a browser client used with PKCE. AgentGateway performs the server-side code exchange, so the sample supplies its client secret to the gateway rather than exposing it to browser code.
support-chat, a confidential client with a client secret used by the console app's password grant.

The browser flow also needs CORS. The sample allows the Admin UI origin, permits the MCP request headers, exposes mcp-session-id, and enables credentials:

mcp:
  policies:
    cors:
      allowOrigins: ['http://localhost:15000']
      allowMethods: [POST, OPTIONS]
      allowHeaders:
        - authorization
        - content-type
        - accept
        - cache-control
        - mcp-protocol-version
        - mcp-session-id
      exposeHeaders: [mcp-session-id]
      allowCredentials: true
      maxAge: 10m

The Admin UI's Apply CORS button writes this policy to the mounted config file. That is why the Compose mount is writable even though the container root filesystem remains read-only. See the official MCP authentication, MCP authorization, and CORS guides.

The console client uses the password grant because a desktop tool can hold a secret:

var form = new FormUrlEncodedContent(
    new Dictionary<string, string>
    {
        ["grant_type"] = "password",
        ["client_id"] = "support-chat",
        ["client_secret"] = "support-chat-secret",
        ["username"] = "alice",
        ["password"] = "alice-password",
    }
);
var tokenResponse = await http.PostAsync(KeycloakTokenUrl, form);

The gateway itself never stores user passwords. It only validates tokens, which keeps credential handling inside the identity provider where it belongs.

MCP authorization: CEL rules

Authentication answers "who are you". Authorization answers "what may you do". The gateway evaluates CEL expressions per tool call:

mcp:
  policies:
    mcpAuthorization:
      rules:
        - 'mcp.tool.target == "tickets"'
        - 'mcp.tool.target == "catalog"'
        - 'mcp.tool.target == "time"'
        - 'mcp.tool.target == "everything"'
        - 'mcp.tool.target == "customers" && "support-admin" in jwt.realm_access.roles'

Rules are OR-ed: a call is allowed if any rule matches. The first four rules let everyone use tickets, catalog, time, and everything tools. The fifth rule narrows the customers tools to users holding the support-admin Keycloak role. Because jwt.realm_access.roles is an array claim populated by Keycloak, the expression reads naturally: Alice passes, Bob is blocked.

The gateway removes unauthorized tools from tools/list responses and returns 403 for blocked calls. Bob can tell that customer tools exist, but customers_customers_get returns 403 for him while Alice gets data. This matters to agent clients: an LLM usually proposes only tools in its list, so hiding a tool also keeps the model from trying it.

The CEL playground

The Admin UI ships a CEL playground at /ui/cel/. It shows the full request context for a sample MCP call, so you can iterate on rules before deploying them. Three expressions worth trying there:

mcp.tool.target == "customers" && "support-admin" in jwt.realm_access.roles

The role check from the config. Paste it and change the simulated role claim to see the result flip.

default(request.headers["x-user-id"], "anonymous")

Header access with a fallback. Useful when your downstream systems need a user identifier even for unauthenticated traffic.

default(jwt.sub, "anonymous")

The subject of the validated JWT, falling back to anonymous. This is exactly the expression the remote rate-limit descriptors use to key per-user limits.

The playground makes the authorization policy testable without restarting the gateway, which is the fastest way to learn the CEL environment.

HTTP authorization is a separate policy layer

MCP authorization decides whether a particular tool invocation is allowed. HTTP authorization protects the request itself and applies to ordinary HTTP, LLM, MCP, and A2A traffic. For mandatory claims, use require because it fails closed when the claim is absent:

policies:
  authorization:
    rules:
      - require: 'jwt.sub != ""'
      - deny: 'request.headers["x-blocked"] == "true"'

This distinction helps keep policies understandable. Use mcpAuthorization for mcp.tool.target and mcp.tool.name; use authorization for paths, headers, and general request claims. The HTTP authorization guide documents the allow, deny, and require evaluation rules.

The LLM service-client endpoint uses the separate API-key authentication policy:

llm:
  policies:
    apiKey:
      mode: strict
      keys:
        - key: sk-alice-abc123def456
          metadata:
            user: alice

That endpoint rejects missing and invalid keys with 401. Browser users should use :4001 and OIDC/PKCE instead of placing one of these keys in browser code.

A2A proxying

Agent-to-Agent (A2A) is how agents talk to other agents. The sample adds an A2A gateway on port 3001 and a route that proxies to the .NET SupportAgent. The route is protected with the same Keycloak JWT validation used by the MCP gateway; browser clients obtain the token through the public Keycloak client with PKCE, while the SupportChat console uses the password grant for simplicity:

gateways:
  a2a:
    port: 3001

routes:
  - name: a2a-support-agent
    gateways: [a2a]
    policies:
      a2a: {}
      cors:
        allowOrigins: ['*']
        allowHeaders: [content-type, cache-control, authorization]

      jwtAuth:
        mode: strict
        issuer: http://localhost:8080/realms/agentgateway
        audiences:
          - agentgateway
        jwks:
          url: http://keycloak:8080/realms/agentgateway/protocol/openid-connect/certs

      localRateLimit:
        - maxTokens: 60
          tokensPerFill: 60
          fillInterval: 1m
          type: requests
    backends:
      - host: support-agent:9999

The agent is built on the a2a-net SDK and uses the same gateway LLM endpoint internally, so even the agent's model traffic passes through the gateway's keys and guardrails:

builder.Services.AddA2AServer(server =>
    server
        .SupportsStreaming()
        .Host(agent =>
            agent
                .WithCard(card =>
                    card.WithName("Support Escalation Agent")
                        .WithDescription("A2A agent that summarizes support tickets.")
                        .WithVersion("1.0.0")
                )
                .UseRuntime<SupportAgentRuntime>()
        )
        .UseMemoryStore()
        .UseMemoryTaskQueue()
        .UseHttpTransport()
);

The runtime streams LLM output back as task events, appending text chunks to one artifact:

await foreach (var content in chatClient.GetStreamingResponseAsync(
    messageText, cancellationToken: cancellationToken))
{
    yield return new A2A.Models.TaskArtifactUpdateEvent
    {
        ContextId = task.ContextId,
        TaskId = task.Id,
        Artifact = new()
        {
            ArtifactId = artifactId,
            Parts = [new A2A.Models.TextPart { Text = content.Text }],
        },
        Append = !isFirstChunk,
    };
}

SupportChat talks to the agent with the A2A HTTP+JSON /v1/message:send endpoint through the gateway and attaches the same Keycloak bearer token it uses for MCP:

client.DefaultRequestHeaders.Authorization =
    new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", accessToken);

var payload = new
{
  message = new
  {
    messageId = requestId,
    role = "user",
    parts = new[] { new { kind = "text", text } },
  },
};
var response = await client.PostAsJsonAsync($"{agentEndpoint}/v1/message:send", payload);

An unauthenticated /v1/message:send now returns 401, so the A2A route is no longer the open backdoor it often becomes in early demos.

The response walk ends at result.task.artifacts[*].parts[*].text. The gateway rewrites the agent card on /.well-known/agent-card.json, so other agents always discover the gateway URL, never the backend container.

Guardrails

The policy shape follows AgentGateway's LLM guardrails documentation.

AgentGateway guardrails configuration for request and response protection

Guardrails run before the upstream call (on the request) and after the model answers (on the response). Both are regex-based in this sample:

llm:
  policies:
    guardrails:
      streaming: Enabled
      request:
        - regex:
            action: reject
            rules:
              - pattern: (?i)(ignore|disregard) (all )?(previous|prior|above) (instructions|prompts|rules|messages)
              - pattern: (?i)(reveal|show) (your|the) (system prompt|system instructions|prompt)
              - pattern: (?i)(you are now|act as if) .*(no restrictions|unfiltered|uncensored)
      response:
        - regex:
            action: reject
            rules:
              - builtin: email
              - builtin: ssn
              - builtin: creditCard

The request rules catch the classic jailbreak family: "ignore all previous instructions", "reveal your system prompt", "act as if you have no restrictions". The response rules use builtin detectors for PII, so a model that leaks an email address or a credit card number gets its response rejected before the client sees it. streaming: Enabled keeps SSE streaming working, which matters because rejection still needs to surface through an active stream.

The verify script proves the request guard end to end: it sends a jailbreak prompt and accepts a reject status, because the exact status depends on how the gateway reports blocked traffic in a given version.

Rate limiting

See the official rate-limit documentation for token buckets, token-based accounting, remote Envoy-compatible limits, and failure behavior.

The default config uses local in-memory token buckets, which need no external service:

llm:
  policies:
    localRateLimit:
      - maxTokens: 60
        tokensPerFill: 1
        fillInterval: 1s
        type: requests
      - maxTokens: 50000
        tokensPerFill: 50000
        fillInterval: 1h
        type: tokens

Two buckets for the LLM gateway: 60 requests per second sustained, plus a 50k token per hour budget. The MCP gateway gets its own bucket of 2,000 requests per minute because one federated session may fan out across six targets. The token bucket shape means short bursts are absorbed while the sustained rate stays capped.

Local rate limits live in the gateway process, so they are per-instance and reset on restart. That is fine for a single-node demo and wrong for exact global accounting across replicas. For per-user limits shared across instances, the config file includes a commented remote variant using an Envoy-compatible rate limit service:

remoteRateLimit:
  host: ratelimit:8081
  domain: agentgateway
  failureMode: failOpen
  type: requests
  descriptors:
    - entries:
        - key: user
          value: 'apiKey.user'

The MCP variant keys on the JWT subject with a fallback:

- key: user
  value: 'default(jwt.sub, "anonymous")'

The matching Envoy server config gives Alice 15 requests per minute and Bob 30, per user, across every gateway replica:

descriptors:
  - key: user
    value: alice
    rate_limit:
      unit: minute
      requests_per_unit: 15
  - key: user
    value: bob
    rate_limit:
      unit: minute
      requests_per_unit: 30

You opt in with ./scripts/start-mcps.sh --ratelimit, which composes in the extra services (the Envoy-compatible server plus Redis) and swaps the config mount to the remote variant. Everything else stays identical, which is the point of keeping the two configs in lockstep.

LLM governance: costs and request bounds

Authentication and rate limiting answer who may call the model and how much traffic may pass through the gateway. Cost attribution answers a third question: what did each request cost? AgentGateway's model cost catalog maps provider/model pairs to token prices and adds realized cost to request logs, traces, metrics, and the Admin UI analytics view.

The sample keeps its pricing data in a separate file so it can be updated without changing the routing policy:

{
  "providers": {
    "deepseek": {
      "models": {
        "deepseek-chat": {
          "rates": { "input": "0.27", "output": "1.10" }
        },
        "deepseek-reasoner": {
          "rates": { "input": "0.55", "output": "2.19" }
        }
      }
    }
  }
}

The rates are USD per one million tokens. The gateway loads the catalog from the static config section and mounts it read-only in Compose:

config:
  modelCatalog:
    - file: /costs/catalog.json

llm:
  models:
    - name: deepseek-chat
      provider: deepseek
      params:
        apiKey: $DEEPSEEK_API_KEY
        model: deepseek-chat

Open the Admin UI at http://localhost:15000/ui/llm/analytics to inspect requests, tokens, and realized cost. The cost dashboard guide explains how the database and catalog work together. For production, use PostgreSQL instead of sharing one SQLite file between gateway instances; see the request log database guide.

Cost visibility does not automatically enforce a budget. Combine the catalog with budget and spend limits and local or remote rate limiting when requests must be rejected after a quota is exhausted. The sample demonstrates the simpler request-time safety bound: every concrete model caps max_tokens at 1024, so callers cannot silently request an unbounded completion:

llm:
  models:
    - name: deepseek-chat
      transformation:
        max_tokens: 'min(llmRequest.max_tokens, 1024)'

This uses an LLM request transformation, documented in the LLM transformation guide. The transformation is attached to both concrete models, so weighted virtual model selection cannot bypass the limit.

Observability

AgentGateway analytics dashboard showing observed LLM traffic

AgentGateway's LLM observability guide lists the request, token, model, provider, and realized-cost fields available to metrics and traces.

The gateway exports OpenTelemetry traces and Prometheus metrics, and writes structured JSON logs to stdout plus a SQLite request log:

config:
  adminAddr: 0.0.0.0:15000
  logging:
    format: json
    database: sqlite:///var/log/agentgateway/request-log.db
  metrics:
    fields:
      add:
        user_id: apiKey.user
  tracing:
    otlpEndpoint: http://otel-collector:4317
    randomSampling: true

There are two telemetry paths in this sample. Traces use OTLP end to end. Logs start as AgentGateway stdout records, then the OpenTelemetry Collector's filelog receiver reads Docker's JSON log files and sends the records to Loki:

Rendering diagram...

flowchart LR
    G[AgentGateway]
    D[DeepSeek or MCP target]
    J[Docker JSON log files]
    C[OpenTelemetry Collector :4317\nOTLP/gRPC receiver]
    T[Tempo :4317\nOTLP/gRPC]
    L[Loki :3100\nOTLP/HTTP]
    F[Langfuse :3000\nOTLP/HTTP endpoint]
    GF[Grafana :13000\nExplore]

    G -->|OTLP/gRPC traces| C
    G --> D
    G -->|structured JSON stdout| J
    J -->|filelog receiver| C
    C -->|OTLP/HTTP| L
    C -->|OTLP/gRPC| T
    C -->|OTLP/HTTP with Basic auth| F
    GF -->|query| T
    GF -->|query| L

Traces: AgentGateway, the collector, and Tempo

AgentGateway is the OTLP client in this deployment. Its tracing section sends trace batches to otel-collector:4317, which is the collector's internal OTLP gRPC listener. Port 4317 is not Tempo's host port and it is not Grafana's query port; it is the ingestion port exposed by the collector:

tracing:
  endpoint: http://otel-collector:4317
  protocol: grpc
  random_sampling: true

The collector receives those OTLP spans once, batches them, and exports a copy to Tempo over the Compose network. Tempo listens for OTLP gRPC on its internal port 4317, while its HTTP query API is exposed as host port 3200:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo]

Grafana does not receive spans. Its provisioned Tempo datasource queries http://tempo:3200 from inside the Compose network. In the browser, use Grafana at http://localhost:13000, open Explore, select Tempo, and search recent traces. The same trace ID can then be opened in Tempo's detail view.

Logs: AgentGateway, the collector, and Loki

AgentGateway's JSON logger writes request records to stdout. Docker stores that stdout in files under /var/lib/docker/containers/*/*-json.log. The Collector reads those files through its filelog/docker receiver. The receiver parses Docker's JSON envelope, moves the application message into the log body, and preserves its timestamp before the logs enter the Collector pipeline:

receivers:
  filelog/docker:
    include:
      - /var/lib/docker/containers/*/*-json.log
    start_at: end
    include_file_path: true
    operators:
      - type: json_parser
        parse_from: body
      - type: move
        from: attributes.log
        to: body
      - type: time_parser
        parse_from: attributes.time
        layout: '%Y-%m-%dT%H:%M:%S.%fZ'

service:
  pipelines:
    logs:
      receivers: [filelog/docker]
      processors: [batch]
      exporters: [otlphttp/loki]

The compose file mounts the Docker log directory read-only into the Collector. This file-based setup matters on Docker Desktop: the socket file can exist inside a container while Docker API discovery still fails. Loki receives the Collector's OTLP/HTTP export at http://loki:3100/otlp; its HTTP query API is exposed on host port 3100, and Grafana queries it through the provisioned Loki datasource.

In Grafana Explore, select Loki and run:

{job="docker"}

The result should contain labels such as filename, job, and stream, followed by the container log lines. If Loki has no labels, inspect the Collector's filelog/docker receiver first; Grafana cannot display logs that Loki never indexed.

The Collector must also mount /var/lib/docker/containers read-only and run with permission to read Docker's root-owned log files. The filelog receiver does not make AgentGateway an OTLP log producer; it converts the existing Docker file records into the Collector's internal log data model. That gives this deployment one place for batching, filtering, redaction, and export, without running two readers against the same files.

LLM traces: the collector and Langfuse

Langfuse is another exporter target for the collector's trace pipeline. The gateway still sends one OTLP trace to collector port 4317; the collector then sends a second copy to Langfuse's OTLP-compatible HTTP endpoint. Langfuse is therefore not listening on the collector's gRPC port in this sample. Its web UI is exposed on host port 13001, while the collector reaches the service at http://langfuse-web:3000/api/public/otel:

exporters:
  otlphttp/langfuse:
    endpoint: http://langfuse-web:3000/api/public/otel
    headers:
      Authorization: 'Basic ${env:LANGFUSE_AUTH}'
      x-langfuse-ingestion-version: '4'

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo, otlphttp/langfuse]

The Basic authentication value is created from the Langfuse public and secret keys and supplied to the collector as LANGFUSE_AUTH. The collector's HTTP exporter adds that header, so application clients do not need Langfuse keys. Langfuse stores the received LLM observations in its Postgres, ClickHouse, and event-storage services, then the web UI at http://localhost:13001 displays model, prompt, token, latency, and cost information.

This gives one request two useful views: Tempo shows the distributed trace and its gateway or MCP spans, while Langfuse shows the LLM-specific observation. The trace ID and timestamps help correlate them. Prometheus is separate again: it scrapes AgentGateway's metrics endpoint instead of receiving OTLP traces. The Grafana provisioning includes Prometheus, Loki, and Tempo datasources plus a prebuilt dashboard that uses the user_id label:

sum by (user_id) (rate(agentgateway_gen_ai_client_requests_total[5m]))
sum by (user_id) (agentgateway_gen_ai_client_token_usage_sum)

Those queries answer the two questions that always get asked in production: how many requests per user, and how many tokens per user.

Langfuse is self-hosted in the same compose file (web, worker, Postgres, Redis) and exposed on port 13001. The Langfuse integration guide describes the same OTLP path for Langfuse Cloud. Because the gateway forwards its internal trace context upstream, a single request shows up in Grafana as a trace with spans for the gateway and the MCP targets, and in Langfuse as an LLM trace with model, prompt, token counts, and catalog-derived cost. Cross-referencing the same request ID between the two views is the fastest way to debug "the model answered weirdly" versus "the gateway dropped it".

The Admin UI on port 15000 also surfaces recent traffic and gateway logs from the SQLite request log, which is handy before you even open Grafana.

Production features outside the runnable sample

The Compose sample is intentionally runnable with one provider and local development credentials. It now includes OpenAPI-to-MCP, retries, mirroring, priority failover, and an opt-in remote token-budget profile. The following sections separate those runnable paths from features that still require external infrastructure; each snippet is grounded in the official documentation.

Provider failover and health-based eviction

The sample's weighted deepseek-smart model distributes traffic, but it is not failover. Failover needs ordered priority groups and a health policy. A provider that returns configured 5xx/429 responses can be evicted temporarily, allowing traffic to move to the next group. See the standalone virtual model guide and failover guide.

llm:
  virtualModels:
    - name: support-model
      routing:
        failover:
          targets:
            - model: primary-model
              priority: 0
            - model: backup-model
              priority: 1

  models:
    - name: primary-model
      health:
        eviction:
          consecutiveFailures: 1
          duration: 60s

Use this when provider availability matters more than fixed weighted traffic. The sample's deepseek-resilient model uses the same structure with a primary and backup DeepSeek target; a real deployment should use an independently provisioned provider for meaningful failover.

Conditional policies and request matching

The sample uses separate endpoints for API keys and browser OIDC. Conditional policies let one route select a transformation, rate limit, authorization rule, or direct response from CEL context. The conditional policy guide documents the supported policy variants.

policies:
  conditional:
    - matches:
        - 'jwt.realm_access.roles.exists(role, role == "support-admin")'
      policies:
        transformation:
          max_tokens: 'min(llmRequest.max_tokens, 2048)'
    - matches:
        - 'request.headers["x-environment"] == "canary"'
      policies:
        directResponse:
          status: 403
          body: canary disabled

This sample does not add conditional policy configuration because its authentication and model paths are deliberately explicit.

Retries, backoff, and timeouts

Retries and per-try timeouts protect short, retryable upstream failures, but blind retries are dangerous for streaming or non-idempotent operations. The retry guide and per-try timeout guide show the route-level controls:

policies:
  retry:
    attempts: 3
    backoff: 500ms
    codes: [429, 500, 503]

The sample enables these retries for MCP traffic. LLM retries should be enabled only after considering buffering, idempotency, and streaming behavior.

Fault injection and mirroring

Fault injection adds delay or an injected failure to test timeout and failover behavior. Mirroring copies traffic to an analysis backend while the primary response remains authoritative. The sample enables 10% MCP mirroring to a side-effect-free local HTTP sink. Fault injection is available in the optional standalone configuration at deploy/optional/fault-injection.yaml.

policies:
  faultInjection:
    delay:
      fixed: 750ms
  requestMirror:
    backend:
      host: evaluation-provider:8080
    percentage: 0.05

See the fault injection and mirroring guides. Treat mirrored payloads as production data: apply redaction, retention, access control, and cost controls before enabling them.

TLS, HTTPS, and mTLS

The sample uses HTTP on a local Docker network. Production listeners should terminate HTTPS and can require client certificates. Backend TLS and mTLS are separate from listener TLS, so protect both the client-to-gateway and gateway-to-provider hops. See the TLS guide, mTLS listener guide, and TLS installation guide.

listeners:
  - name: secure-llm
    address: 0.0.0.0:8443
    tls:
      certificate: /etc/agentgateway/tls/tls.crt
      privateKey: /etc/agentgateway/tls/tls.key
      clientCA: /etc/agentgateway/tls/ca.crt
      requireClientCertificate: true

Mount certificates as secrets, rotate them through the platform, and never commit private keys to the sample repository.

OpenAPI to MCP and code mode

The five native MCP targets in this sample are MCP servers. AgentGateway can also expose operations from an OpenAPI document as MCP tools, mapping unique operationId values to tool names. See the OpenAPI-to-MCP guide.

mcp:
  targets:
    - name: orders-api
      openapi:
        schema:
          url: https://orders.example.com/openapi.json
        host: orders.example.com

The sample includes a local Petstore schema and service under deploy/openapi/. Validate schemas, operation IDs, upstream authentication, and tool argument limits before exposing an API to an agent. Code mode is an additional JavaScript sandbox pattern for composing several generated operations; see the code mode article.

Separate MCP paths and per-target authentication

Federation through one /mcp endpoint is useful for the demo, but separate paths can isolate consumers and policies. Backend authentication injects upstream credentials without exposing them to MCP clients. See backend authentication.

routes:
  - name: orders-mcp
    matches:
      - path:
          pathPrefix: /orders/mcp
    policies:
      mcpAuthentication:
        mode: strict
    backends:
      - mcp:
          host: https://orders.example.com/mcp
          policies:
            backendAuth:
              secretRef: orders-api-token

The sample uses one shared gateway MCP policy and public demo services; it does not include external secret references or per-target credentials.

MCP-native guardrails with ExtMCP

The sample's regex and builtin PII guards protect LLM request and response payloads. ExtMCP is the MCP-native extension point for inspecting or mutating tools/list, tools/call, and related MCP messages in an external gRPC processor. Processors can fail open or closed and run in a defined order. See the ExtMCP guardrails overview and setup guide.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
spec:
  targetRefs:
    - group: agentgateway.dev
      kind: AgentgatewayBackend
      name: mcp-backend
  backend:
    mcp:
      guardrails:
        processors:
          - remote:
              backendRef:
                name: ext-mcp
                port: 4445
              failureMode: FailClosed
              methods:
                tools/call: Request
                tools/list: Response

The sample includes the Kubernetes policy fragment at deploy/optional/mcp-guardrails-policy.yaml, but no ExtMCP server because it requires a separate protocol implementation and service lifecycle.

External moderation providers

Regex rules are deterministic and local. For semantic prompt-injection or content moderation, layer an external moderation service through the guard flow or an external processor. The prompt guard overview describes request guards, response guards, webhooks, and external moderation.

llm:
  models:
    - name: '*'
      provider: openAI
      params:
        model: gpt-4o-mini
        apiKey: '$OPENAI_API_KEY'
      guardrails:
        request:
          - openAIModeration:
              model: omni-moderation-latest
              policies:
                backendAuth:
                  key: '$OPENAI_API_KEY'
                rejection:
                  body: 'Content blocked by moderation policy'

The default Compose profile stays provider-neutral and uses regex and builtin detectors. The optional sample fragment at deploy/optional/moderation-policy.yaml enables OpenAI moderation when OPENAI_API_KEY is available.

Additional LLM API types

The sample exercises /v1/chat/completions only. The gateway documentation also covers /v1/responses, /v1/messages, /v1/embeddings, /v2/rerank, /v1/models, and token-counting endpoints. See API types, embeddings, and LLM overview.

curl http://localhost:4000/v1/embeddings \
  -H "Authorization: Bearer $ALICE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"embedding-model","input":"ticket text"}'

This is an API-shape example only: the current sample does not configure an embedding backend or test this request.

Kubernetes catalogs and PostgreSQL

The sample mounts a local JSON catalog and writes a SQLite request log. A Kubernetes deployment can store catalog JSON in a same-namespace ConfigMap; multiple catalogs merge in order, with later entries overriding earlier ones. For replicated production deployments, use the documented database guidance before selecting PostgreSQL, migrations, backups, and HA behavior. See the Kubernetes cost catalog guide, database integration guide.

apiVersion: v1
kind: ConfigMap
metadata:
  name: model-costs
data:
  catalog.json: |
    { "providers": { "deepseek": { "models": {} } } }

The runnable sample keeps Compose credentials in .env for local development only. Production deployments should keep provider credentials in the platform secret manager and outside gateway configuration.

IDE and assistant integrations

The requested VS Code, GitHub Copilot, and Claude Code integrations are not documented as first-class standalone AgentGateway integrations in the current official material. The sample therefore exposes standard HTTP endpoints and does not claim a native plugin. Use each client's supported OpenAI-compatible base URL and credential configuration only after validating its API shape, streaming behavior, and authentication requirements. Track upstream integration work in the AgentGateway issue tracker.

Verification

Two verification layers come with the sample.

Automated smoke test

scripts/verify.sh runs nine checks against the running stack, each mapped to a gateway feature:

Keycloak token retrieval for Alice and Bob.
LLM auth: a valid virtual key returns 200, an invalid key returns 401.
MCP auth: a call without a token returns 401.
Tool multiplexing: tools/list contains prefixed tools from all six targets, including openapi_getInventory.
CEL authorization: Bob gets 403 on customers_customers_get, Alice gets 200.
Guardrails: a jailbreak prompt is rejected.
Rate limits: a burst of 70 requests produces a 429.
A2A auth: the agent card returns 401 without a token and 200 with Alice's token.
Metrics: the Prometheus endpoint exposes the user_id label.

Browser-only checks cover redirect and cookie behavior that ordinary bearer token tests cannot cover:

# LLM OIDC/PKCE flow: opens Keycloak login in a browser
start http://localhost:4001/v1/models

# MCP protected-resource metadata
curl http://localhost:3000/.well-known/oauth-protected-resource/mcp

# MCP authorization-server metadata, including S256 support
curl http://localhost:3000/.well-known/oauth-authorization-server/mcp

# CORS preflight used by the Tool Playground
curl -i -X OPTIONS http://localhost:3000/mcp \
  -H "Origin: http://localhost:15000" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: authorization,content-type,accept,mcp-session-id"

The LLM browser flow should look like this:

GET /v1/models              -> 302 to Keycloak with code_challenge=S256
Keycloak login              -> authorization code
GET /oauth/callback         -> 302 back to /v1/models
GET /v1/models with cookie  -> authenticated gateway request

Each check prints PASS or FAIL and the script exits non-zero if anything fails:

./scripts/verify.sh

==> [4/8] Tool multiplexing
  PASS: got 10 tools
  PASS: tools prefixed by target (tickets_tickets_list, everything_echo, ...)
==> [5/8] CEL authorization
  PASS: bob blocked from customers (HTTP 403)
  PASS: alice allowed on customers (HTTP 200)

There is also a .NET test project (tests/AgentGateway.Samples.Tests) using xUnit v3 and Shouldly. The tests are integration tests that call the live gateway; when the stack is down they skip cleanly, and when it is up they cover LLM virtual-key auth, MCP multiplexing, CEL authorization (Alice versus Bob), A2A JWT auth, request guardrails, local rate limiting, and the Admin UI / metrics endpoints:

dotnet test tests/AgentGateway.Samples.Tests/AgentGateway.Samples.Tests.csproj

Manual walkthrough in the Admin UI

Open http://localhost:15000/ui/. The Gateway Overview lists LLM, MCP, and Traffic capabilities.
Traffic > Routes confirms the API-key LLM route on port 4000, OIDC/PKCE LLM route on port 4001, MCP route on port 3000, and protected a2a-support-agent route on port 3001.
LLM > Client Setup picks deepseek-smart and an sk-alice-* key and hands you a ready-to-run curl snippet, which checks virtual-key auth and virtual-model routing.
Open http://localhost:4001/v1/models and sign in as Alice. Confirm the redirect contains code_challenge_method=S256, then confirm the callback returns to the protected route with an AgentGateway session cookie.
CEL playground at /ui/cel/ evaluates authorization expressions against a sample request context.
MCP > Tool Playground should show browser access enabled. Initialize through Keycloak OAuth/PKCE and call tickets_tickets_list; log in as Bob and try customers_customers_get to see the 403.
MCP > connected targets lists all six targets and their health, including OpenAPI Petstore.
A2A - the agent card endpoint (/.well-known/agent-card.json) on port 3001 requires the same Keycloak JWT. Message requests use /v1/message:send; an unauthenticated request returns 401.
Logs/Traffic shows recent traffic; cross-check the same request IDs in Grafana and Langfuse.

AgentGateway MCP Tool Playground for initializing sessions and calling tools

MCP Inspector walkthrough

The Admin UI is useful for checking browser CORS and PKCE. MCP Inspector is useful for testing the gateway as an MCP client over Streamable HTTP. The sample includes a repeatable CLI check:

export KEYCLOAK_TOKEN="<Alice access token>"
./scripts/inspector-smoke.sh

The script uses the official Inspector CLI and verifies three behaviors:

tools/list returns native prefixed tools and the OpenAPI-generated openapi_getInventory tool.
tools/call reaches the Docker-hosted tickets MCP server.
tools/call reaches the Petstore REST service through the OpenAPI adapter.

For visual evidence, run the Inspector UI while Compose is running:

npx @modelcontextprotocol/inspector

Select Streamable HTTP, enter http://localhost:3000/mcp, and add an Authorization header with Alice's Keycloak bearer token. Capture one tools/list screen and one successful openapi_getInventory result. Repeat the same calls in the Admin UI MCP Tool Playground to verify CORS and the browser OAuth/PKCE path. The repository's screenshot evidence should show the aggregated tool list, the OpenAPI tool, and a successful tool result rather than only the Inspector landing page.

Conclusion

An AI gateway moves repeated security and operational work out of application code. In this sample, one AgentGateway instance fronts DeepSeek with virtual keys and a weighted virtual model. It also multiplexes six MCP targets behind one authenticated endpoint, evaluates CEL rules against Keycloak tokens, rejects prompt-injection attempts and PII leaks, rate-limits users with an optional Envoy-compatible remote setup, proxies an A2A agent, and exports traces, metrics, and logs to Grafana and Langfuse.

The same setup can grow with the application. Swap DeepSeek for another provider, add or remove MCP targets without changing clients, tighten a CEL rule without redeploying, and move from local to remote rate limits when the stack needs more than one replica.

The full sample, including the gateway config, Compose stack, .NET MCP servers, A2A agent, Keycloak realm, verification script, and xUnit v3 test project, is available in blog-samples/agentgateway-ai-gateway. Start it with ./scripts/start-mcps.sh, run ./scripts/verify.sh, execute the .NET tests with dotnet test, and try changing the guardrail rules in the CEL playground. Those steps show exactly what the gateway checks on each request.