The Role of Virtual MCP in Managing LLM Costs

July 7, 2026

Eitan Suez

Agentic workflows can multiply LLM costs. Explore how Virtual MCP in agentgateway simplifies MCP management while reducing token consumption through centralized tool discovery, filtering, and governance.

Introduction

When working with LLMs it's important to be aware of the costs involved, and to have controls in place. We need to be able to monitor costs by team, and to put in place policies to control costs, such as rate limiting. It's important to centrally control access and to be able to set distinct budgets for different teams. Such policies would allow certain teams who play a more critical function to continue operating even after other teams' budgets have been exhausted.

Providers price access to LLMs by number of input and output tokens used in interactions with models. Frontier models of course carry a premium price. One strategy is to select the least priced model that can do the job in question. Another can involve falling back to a second tier model when the budget for the first tier model is exhausted.

But beyond the basics of individuals' interactions with LLMs, these days our interactions with LLMs often happen through agents. It's therefore important to understand the agentic loop, and its implications on the cost of interacting with LLMs.

The agentic loop

When a user interacts with an agent, the agent acts as a type of middle man, in two specific ways:

Making calls to LLMs on behalf of the user.
Making calls to MCP tool servers on behalf of the LLM.

A scenario could be described as follows:

A user sends a message to the agent, and the agent turns around and makes a call to the LLM. The input sent to the LLM (the context) during the initial call is typically minimal; it includes the user's message, the system prompt, and the list of tools and their descriptions (schema) which convey to the LLM what tools it can call and how exactly to call them.

The size of that context translates to a specific monetary cost, though the initial context is relatively small, and can be dominated by the tools list, depending on the number of tools that the agent was configured with.

The LLMs response, aside from its reasoning and thoughts about the task, will typically involve a request to make tool calls. These all count towards the output tokens generated. The agent turns around and calls the tools and captures the tool call responses. Interactions between agents and MCP servers do not involve any AI costs.

The agent turns back to the LLM, and returns not only the results from the tool calls, but also the conversation history, the prompts, the tools list and descriptions (again). Certain tools by their very nature can return a large amount of information, ones that fetch the contents from web pages or database queries results. And so with every turn, the context accumulates and the cost compounds.

In the context of agentic loops, we see AI costs grow dramatically. Compared to a "straight" conversation with a model, a multi-turn agentic interaction could easily cost an order of magnitude (10x) higher.

Central control with agentgateway

Agentgateway is a modern proxy designed both for traditional use cases involving microservices, and expressly for agentic workloads: LLMs, MCP servers, and agents. In an agentic setup, it's common to use agentgateway to proxy both LLM calls and calls to MCP servers. It can be deployed standalone or on Kubernetes, and comes with a long list of features specifically designed for managing interactions with LLMs, enabling monitoring, and the management and control of costs through budgets, quotas, and limits. agentgateway also supports security, authentication and authorization and the control of access to models, agents, and tools.

Agentgateway's Virtual MCP feature

Virtual MCP, also known as MCP Multiplexing, is a capability that allows for the federation of multiple MCP servers behind a single proxy.

Through federation, agentgateway becomes the target MCP server for clients, and provides a single point where access to all these MCP servers can be monitored and controlled. agentgateway maps incoming requests for specific tool calls to their corresponding backend MCP server, manages sessions with the backend MCP servers, and relays responses.

Advantages that derive from the ability to perform MCP server federation include:

Client configuration is simplified: clients point to a single proxy and have access to multiple sets of tools from disparate MCP servers.
Clients are insulated from changes to the backend configuration: we can federate an additional server, update the address of a backend MCP server, or have the option to host an MCP server locally or call a remote MCP server.
Centralized MCP observability and metrics: monitor calls to MCP servers, distribution of calls by client, tool call response sizes.
Centralized control: policies can be applied at the proxy to control what specific tools are exposed to which clients.

MCP federation example

Below is an example of agentgateway running on Kubernetes, although agentgateway can also be provisioned standalone (on VMs or in Docker for example).

Provisioning a gateway on Kubernetes is as simple as applying the Kubernetes Gateway API's Gateway resource:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: mcp-proxy
spec:
  gatewayClassName: agentgateway
  listeners:
  - name: http
    port: 3000
    protocol: HTTP

Imagine a DevOps scenario, where we are building agents that can help with diagnosing and fixing issues in a target environment by submitting pull requests to a git repository and triggering ArgoCD to control the deployment of fixes to the target environment.

The setup would require MCP servers for ArgoCD and the version control system (GitHub, GitLab, etc..).

In agentgateway, we can multiplex both MCP servers by defining a single, composite backend:

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: composite-mcp-backend
spec:
  mcp:
    targets:
    - name: argo-mcp-server
      static:
        host: argo-mcp-server.argocd.svc.cluster.local
        port: 3000
    - name: gitea-mcp-server
      static:
        host: gitea-mcp-server.gitea.svc.cluster.local
        port: 8080

In the above example, the MCP servers are deployed locally on Kubernetes, but that doesn't have to be the case. We can choose whichever (SaaS or hosted) configuration is simpler to manage, or whichever has the lowest cost profile.

Next, we define a route on the gateway to the composite backend:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: devops-mcp-route
spec:
  parentRefs:
  - name: mcp-proxy
    sectionName: http
  rules:
  - backendRefs:
    - group: agentgateway.dev
      kind: AgentgatewayBackend
      name: composite-mcp-backend

With the gateway provisioned and the route in place, the agentgateway instance URL becomes the federated MCP server endpoint: http://mcp-proxy.default.svc.cluster.local:3000.

We can now proceed to configure agents with the agentgateway as its MCP server. Its tools list is now a combination (aggregation) of all the tools exposed across all backend MCP servers.

Virtual MCP's role in cost management

Although not a standalone cost-control feature, Virtual MCP is a key enabling component in a cost-control strategy.

It gives you the flexibility and single control plane to apply cost-related policies effectively across a heterogeneous set of MCP servers. Without multiplexing, you'd have a harder time getting consistent observability, policies, and governance across many separate endpoints.

The operational simplicity of dealing with a single endpoint for access to multiple MCP servers lowers indirect costs.

When combined with agentgateway policies and authorization scoping, Virtual MCP can have a direct impact on token efficiency in the agentic loop, and help prevent runaway usage.

Below is an example AgentGatewayPolicy (taken from the documentation) illustrating the ability to control what specific tools can be accessed as a function of the identity of the caller or any of the many other MCP metadata information that agentgateway exposes.

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: jwt-rbac
  namespace: agentgateway-system
spec:
  targetRefs:
    - group: agentgateway.dev
      kind: AgentgatewayBackend
      name: github-mcp-backend
  backend:
    mcp:
      authorization:
        action: Allow
        policy:
          matchExpressions:
            - 'jwt.sub == "alice" && mcp.tool.name == "get_me"'

Agentgateway utilizes the Common Expression Language (CEL), and its administrative UI also comes with a CEL sandbox for testing your expressions.

When configured in this way, agentgateway, acting as a federated MCP server will only advertise (i.e. filter) a subset of the backing tools matching the CEL expression.

The impact on AI costs can be dramatic: long lists of MCP tools and their schema definitions is often a dominant component of the context sent to LLMs in agentic systems.

Costs can be managed using a multiplicity of strategies. Agentgateway enterprise has additional compelling features such as search mode and code mode which can be brought to bear to reduce the size of the context even further.