MCP Token Analyzer

About

mcp-token-analyzer is a CLI tool designed to connect to a Model Context Protocol (MCP) server, retrieve its tool definitions and schema, and analyze the token usage of each tool. This tool assists developers in optimizing their tool descriptions and schemas for efficient context usage with LLMs. It provides similar data to the /context slash command in claude code, except it's claude agnostic and specific to analyzing MCP servers.

Example analyzing prometheus-mcp-server:

~/go/src/github.com/tjhop/mcp-token-analyzer (main [ ]) -> ./mcp-token-analyzer --mcp.transport http --mcp.url "http://localhost:8080/mcp"
Tool Analysis (sorted by total tokens)
┌───────────────────────┬───────────────────┬──────┬──────┬────────┬────────┬────────┬───────┐
│        Server         │       Tool        │ Name │ Desc │ Schema │ Output │ Annot. │ Total │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ label_values      │ 2    │ 16   │ 159    │ 0      │ 11     │ 188   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ range_query       │ 2    │ 8    │ 156    │ 0      │ 11     │ 177   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ label_names       │ 2    │ 18   │ 138    │ 0      │ 11     │ 169   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ series            │ 1    │ 6    │ 146    │ 0      │ 11     │ 164   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ exemplar_query    │ 4    │ 9    │ 132    │ 0      │ 13     │ 158   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ delete_series     │ 2    │ 11   │ 110    │ 0      │ 12     │ 135   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ query             │ 1    │ 8    │ 91     │ 0      │ 11     │ 111   │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ targets_metadata  │ 2    │ 10   │ 68     │ 0      │ 11     │ 91    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ docs_search       │ 2    │ 14   │ 49     │ 0      │ 11     │ 76    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ metric_metadata   │ 2    │ 11   │ 48     │ 0      │ 11     │ 72    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ snapshot          │ 1    │ 29   │ 30     │ 0      │ 12     │ 72    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ docs_read         │ 2    │ 15   │ 34     │ 0      │ 11     │ 62    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ ready             │ 1    │ 23   │ 10     │ 0      │ 12     │ 46    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ reload            │ 1    │ 19   │ 10     │ 0      │ 12     │ 42    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ clean_tombstones  │ 4    │ 14   │ 10     │ 0      │ 13     │ 41    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ quit              │ 1    │ 15   │ 10     │ 0      │ 11     │ 37    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ tsdb_stats        │ 3    │ 10   │ 10     │ 0      │ 12     │ 35    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ healthy           │ 1    │ 12   │ 10     │ 0      │ 11     │ 34    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ list_rules        │ 2    │ 10   │ 10     │ 0      │ 11     │ 33    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ alertmanagers     │ 3    │ 7    │ 10     │ 0      │ 12     │ 32    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ wal_replay_status │ 4    │ 5    │ 10     │ 0      │ 13     │ 32    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ docs_list         │ 2    │ 7    │ 10     │ 0      │ 11     │ 30    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ list_targets      │ 2    │ 6    │ 10     │ 0      │ 11     │ 29    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ list_alerts       │ 3    │ 4    │ 10     │ 0      │ 11     │ 28    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ runtime_info      │ 2    │ 4    │ 10     │ 0      │ 11     │ 27    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ build_info        │ 2    │ 4    │ 10     │ 0      │ 11     │ 27    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ flags             │ 1    │ 3    │ 10     │ 0      │ 11     │ 25    │
├───────────────────────┼───────────────────┼──────┼──────┼────────┼────────┼────────┼───────┤
│ prometheus-mcp-server │ config            │ 1    │ 3    │ 10     │ 0      │ 10     │ 24    │
└───────────────────────┴───────────────────┴──────┴──────┴────────┴────────┴────────┴───────┘

Resource Analysis (sorted by total tokens)
┌───────────────────────┬─────────────────────────────────────────────────┬──────┬─────┬──────┬───────┐
│        Server         │                    Resource                     │ Name │ URI │ Desc │ Total │
├───────────────────────┼─────────────────────────────────────────────────┼──────┼─────┼──────┼───────┤
│ prometheus-mcp-server │ Official Prometheus Documentation               │ 3    │ 8   │ 15   │ 26    │
├───────────────────────┼─────────────────────────────────────────────────┼──────┼─────┼──────┼───────┤
│ prometheus-mcp-server │ List of Official Prometheus Documentation Files │ 6    │ 4   │ 15   │ 25    │
├───────────────────────┼─────────────────────────────────────────────────┼──────┼─────┼──────┼───────┤
│ prometheus-mcp-server │ TSDB Stats                                      │ 3    │ 6   │ 9    │ 18    │
├───────────────────────┼─────────────────────────────────────────────────┼──────┼─────┼──────┼───────┤
│ prometheus-mcp-server │ Targets                                         │ 1    │ 4   │ 10   │ 15    │
├───────────────────────┼─────────────────────────────────────────────────┼──────┼─────┼──────┼───────┤
│ prometheus-mcp-server │ List metrics                                    │ 2    │ 5   │ 3    │ 10    │
└───────────────────────┴─────────────────────────────────────────────────┴──────┴─────┴──────┴───────┘

Token Analysis Summary
┌───────────────────────┬──────────────┬───────┬─────────┬───────────┬──────────────┐
│      MCP Server       │ Instructions │ Tools │ Prompts │ Resources │ Total Tokens │
├───────────────────────┼──────────────┼───────┼─────────┼───────────┼──────────────┤
│ prometheus-mcp-server │ 4,527        │ 1,997 │ 0       │ 94        │ 6,618        │
├───────────────────────┼──────────────┼───────┼─────────┼───────────┼──────────────┤
│         TOTAL         │    4,527     │ 1,997 │    0    │    94     │    6,618     │
└───────────────────────┴──────────────┴───────┴─────────┴───────────┴──────────────┘

Features

Transport Support
- stdio: Execute a local binary
- http: Connect to a streaming HTTP endpoint
- streamable-http: Alias for http, used by some MCP clients (Continue, Roo Code, Anthropic MCP Registry)
Configuration File Support
- Load server configurations from mcp.json files
- Compatible with Claude Desktop, Cursor, VS Code, and Continue formats
- Analyze multiple MCP servers in parallel
- Filter to a single server with --server
Comprehensive MCP Analysis
- Server Instructions: Token count for server-level instruction text
- Tools: Token breakdown for names, descriptions, and input schemas
- Prompts: Token breakdown for names, descriptions, and arguments
- Resources & Templates: Token breakdown for names, URIs, and descriptions
Token Counting
- Uses tiktoken via tiktoken-go (defaults to cl100k_base / GPT-4)
- Configurable tokenizer model via --tokenizer.model
- See Supported Tokenizer Models for available models and encodings
Reporting
- Summary table showing token usage per server
- Detail tables showing per-component token breakdowns (always shown for single-server, opt-in via --detail for multi-server)
- Context window percentage calculation via --limit

Installation and Usage

Go

With a working go environment, the tool can be installed like so:

go install github.com/tjhop/mcp-token-analyzer@latest
/path/to/mcp-token-analyzer <flags>

System Packages

Download a release appropriate for your system from the Releases page.

# install system package (example assuming Debian based)
apt install /path/to/package

Note: While packages are built for several systems, there are currently no plans to attempt to submit packages to upstream package repositories.

Building from Source

make build
./mcp-token-analyzer --help

Security and Authentication

OAuth 2.0 and custom TLS configuration fields are parsed from config files but not yet implemented. When present, a warning is emitted. See the ROADMAP for planned authentication support.

Development

Development Environment with Devbox + Direnv

If you use Devbox and Direnv, then simply entering the directory for the repo should set up the needed software.

Building

The included Makefile has several targets to aid in development:

~/go/src/github.com/tjhop/mcp-token-analyzer (main [ ]) -> make

Usage:
  make <target>

Targets:
  help                           print this help message
  tidy                           tidy modules
  fmt                            apply go code style formatter
  lint                           run linters
  binary                         build a binary
  build                          alias for `binary`
  build-all                      test release process with goreleaser, does not publish/upload
  test                           run tests

Configuration Files

The tool can load MCP server configurations from JSON files, supporting formats used by Claude Desktop, Cursor, VS Code, and Continue.

Supported Formats

Claude Desktop / Cursor format (mcpServers key):

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@anthropic/mcp-server-filesystem", "/tmp"],
      "env": { "DEBUG": "true" }
    },
    "remote-api": {
      "url": "http://localhost:3000/mcp",
      "headers": { "Authorization": "Bearer token123" }
    }
  }
}

VS Code format (servers key):

{
  "servers": {
    "postgres": {
      "command": "npx",
      "args": ["-y", "@anthropic/mcp-server-postgres"]
    }
  }
}

Continue format (mcpServers with explicit streamable-http type):

{
  "mcpServers": {
    "remote-api": {
      "type": "streamable-http",
      "url": "http://localhost:3000/mcp"
    }
  }
}

Both mcpServers and servers keys can be present in the same file; mcpServers takes precedence for duplicate server names.

Server Configuration Options

| Field | Description | |-------|-------------| | type | Transport type (optional, inferred from command or url if omitted) | | command | Command to execute (stdio transport) | | args | Command arguments (stdio transport) | | url | Server URL (http transport, must be http:// or https://) | | env | Environment variables for the process | | envFile | Path to .env file (relative to config file location) | | headers | HTTP headers for requests (http transport) |

The transport type (stdio or http) is automatically inferred from the presence of command or url fields. The streamable-http type is accepted and normalized to http.

Multi-Server Output

When analyzing multiple servers, the tool displays a summary table showing token usage across all servers. Use --detail to additionally display per-component detail tables (tools, prompts, resources) with entries from all servers sorted by total tokens. Each row includes the server name so components can be traced back to their origin.

When analyzing a single server (either ad-hoc via CLI flags or via --server), detail tables are always shown automatically.

Supported Tokenizer Models

The --tokenizer.model flag accepts any model name recognized by tiktoken-go. The model name determines which encoding (tokenization scheme) is used for counting. The default is gpt-4 (cl100k_base).

Encodings and Models

| Encoding | Models | |----------|--------| | o200k_base | gpt-4.5, gpt-4.1, gpt-4o (and dated variants like gpt-4o-2024-05-13) | | cl100k_base | gpt-4, gpt-3.5-turbo (and dated variants), text-embedding-ada-002, text-embedding-3-small, text-embedding-3-large | | p50k_base | text-davinci-003, text-davinci-002, code-davinci-002, code-cushman-002 | | r50k_base | text-davinci-001, text-curie-001, text-babbage-001, text-ada-001, davinci, curie, babbage, ada |

You can also pass an encoding name directly (e.g., --tokenizer.model o200k_base) if you prefer to specify the encoding rather than a model name.

Limitations

Anthropic Claude models are not supported by tiktoken-go. There is no official tokenizer for Claude models. When analyzing MCP servers used with Claude, the token counts are approximate. Using o200k_base or cl100k_base provides a reasonable estimate but will not match Claude's actual tokenization.
tiktoken-go's model list reflects OpenAI's public models. Newer or experimental models may not be recognized until the library is updated.

Command Line Flags

usage: mcp-token-analyzer [<flags>]

Flags:
  -h, --[no-]help                Show context-sensitive help (also try
                                 --help-long and --help-man).
  -t, --mcp.transport=stdio      Transport to use (stdio, http, streamable-http)
  -c, --mcp.command=MCP.COMMAND  Command to run (for stdio transport)
  -u, --mcp.url=MCP.URL          URL to connect to (for http transport)
  -m, --tokenizer.model="gpt-4"  Tokenizer model to use (e.g. gpt-4,
                                 gpt-3.5-turbo)
  -f, --config=CONFIG            Path to mcp.json config file
  -s, --server=SERVER            Analyze only this named server from config
      --[no-]detail              Show detailed per-server tables
      --limit=LIMIT              Context window limit for percentage calculation

MCP Servers

MCP Token Analyzer

About

Features

Installation and Usage

Go

System Packages

Building from Source

Security and Authentication

Development

Development Environment with Devbox + Direnv

Building

Configuration Files

Supported Formats

Server Configuration Options

Multi-Server Output

Supported Tokenizer Models

Encodings and Models

Limitations

Command Line Flags

安装命令（包未发布）

Cursor 配置 (mcp.json)

Prometheus MCP Server

MCP Token Analyzer

About

Features

Installation and Usage

Go

System Packages

Building from Source

Security and Authentication

Development

Development Environment with Devbox + Direnv

Building

Configuration Files

Supported Formats

Server Configuration Options

Multi-Server Output

Supported Tokenizer Models

Encodings and Models

Limitations

Command Line Flags

安装命令 （包未发布）

Cursor 配置 (mcp.json)

Prometheus MCP Server

安装命令（包未发布）