The Unseen Engine of Modern AI Agents: A Deep Dive into JSON-RPC

Open any modern AI agent, watch it call a tool, and underneath the abstraction you’ll find a request that looks almost boringly familiar: a method name, some params, an id. That’s JSON-RPC — a protocol older than most of the frameworks built on top of it — and it has quietly become the wire format of the agent era. As we move from monolithic models to agents that use tools, hit live APIs, and query databases, the old “N×M integration problem” comes roaring back: how do you connect N models to M tools without a brittle web of custom code? JSON-RPC, and the MCP standard built on it, is a big part of the answer.

Understanding JSON-RPC

At its core, JSON-RPC is about one thing: calling a method on a remote server. It’s designed to be simple, with only a few key components:

• Request Object: Sent from the client to the server to execute a method. It contains the jsonrpc version, the method to be called, params for the method, and an id to correlate the request with a response.

• Response Object: Sent from the server back to the client. It includes the jsonrpc version, the result of the method call, and the id of the original request.

• Error Object: If a request fails, the server sends a response with an error object instead of a result object.

• Notification: A special type of request that doesn’t expect a response (it has no id). This is useful for “fire-and-forget” messages.

Compared to alternatives like REST, which is resource-oriented, JSON-RPC is action-oriented. Compared to gRPC, which is a high-performance binary protocol, JSON-RPC is human-readable and easier to debug, though it comes with the overhead of text-based JSON.

Model Context Protocol (MCP)

The Model Context Protocol (MCP), initiated by Anthropic, is a prime example of JSON-RPC in action. It’s an open standard designed to help AI models interact with external tools and data sources.

The Problem: LLMs often need access to real-time information or specialized tools (like a calculator or a database) to answer user queries effectively. Managing this “context” can be complex.

The MCP Solution: MCP provides a standardized way for an AI application (the “host”) to communicate with a server that manages these external tools and resources. It uses JSON-RPC for this communication.

MCP Workflow

Here’s a typical workflow for how MCP uses JSON-RPC to enrich an AI model’s context:

1. User Prompt: A user sends a query to an LLM application, like “What were our sales in Q2?”.
2. Tool Discovery: The application (MCP host) determines it needs external data. It sends a JSON-RPC request like method: "get_tools" to the MCP server to see what tools are available.
3. Tool Invocation: The application then sends another JSON-RPC request to invoke the appropriate tool, for example, method: "invoke_tool", with parameters specifying the “sales_database” tool and the query “Q2 sales”.
4. Response Handling: The MCP server executes the tool, retrieves the data, and sends it back in a JSON-RPC response.
5. Context Enrichment: The application uses this data to enrich the LLM’s context, allowing it to generate an accurate answer for the user.

Example MCP Request (invoke_tool):

{
  "jsonrpc": "2.0",
  "method": "invoke_tool",
  "params": {
    "tool_id": "sales_database_tool",
    "parameters": {
      "query": "Q2 sales"
    }
  },
  "id": "request-123"
}

Example MCP Response:

{
  "jsonrpc": "2.0",
  "result": {
    "output": {
      "sales_figures": {
        "Q2_total": 5.2,
        "currency": "USD",
        "unit": "millions"
      }
    }
  },
  "id": "request-123"
}

Practical Workflow: LLM to Tool via MCP

sequenceDiagram
    participant User
    participant Host as Host (LLM)
    participant Client as MCP Client
    participant Server as MCP Server (Jira)

    User->>Host: "Create a new bug report for the login issue."
    Host->>Client: Request available tools
    Client->>Server: JSON-RPC Request: {"method": "tools/list", "id": 1}
    Server-->>Client: JSON-RPC Response: {"result": [{"name": "jira/createIssue",...}], "id": 1}
    Client-->>Host: Provides list of available tools
    Host->>User: "To do this, I need to use the Jira tool. Is that okay?"
    User-->>Host: "Yes, go ahead."
    Host->>Client: Request to invoke jira/createIssue with parameters
    Client->>Server: JSON-RPC Request: {"method": "tools/call", "params": {"name": "jira/createIssue", "arguments": {"title": "Login Fails",...}}, "id": 2}
    activate Server
    Server->>Server: Calls internal Jira API
    Server-->>Client: JSON-RPC Response: {"result": {"success": true, "issueId": "PROJ-123"}, "id": 2}
    deactivate Server
    Client-->>Host: Provides invocation result
    Host->>User: "I've created the issue for you. The ticket is PROJ-123."

Agent-to-Agent (A2A) Protocols

As AI systems become more sophisticated, we’re seeing the rise of multi-agent systems, where multiple AI agents collaborate to solve complex problems. This requires a standardized way for agents to communicate with each other. While there are several emerging protocols in this space, many leverage the principles of RPC, and JSON-RPC is a natural fit.

The Problem: How do you get a “researcher” agent to hand off its findings to a “summarizer” agent in a reliable and standardized way?

The A2A Solution: A2A protocols define the “language” that agents use to talk to each other. For example, the Agent Communication Protocol (ACP) from IBM, while REST-based, is designed to be complementary to MCP. An ACP agent can be exposed as a “tool” within an MCP environment, allowing an MCP-based orchestrator agent to communicate with it using JSON-RPC.

A2A Workflow (via MCP)

Here’s how an orchestrator agent might use a specialized agent via an MCP-to-ACP bridge:

1. Task Decomposition: A user gives a complex task to an orchestrator agent, like “Research the latest trends in renewable energy and write a blog post about it.”
2. Agent Invocation: The orchestrator agent, acting as an MCP host, sends a JSON-RPC request to invoke a “researcher” agent (which is exposed as a tool).
3. Task Execution: The researcher agent (which might be running on ACP) receives the request, performs its research, and returns the findings.
4. Handoff: The orchestrator agent then sends another JSON-RPC request, this time to a “writer” agent, passing the research findings as a parameter.
5. Final Output: The writer agent drafts the blog post and returns it to the orchestrator, which then presents it to the user.

Example A2A Request (invoking a summarizer agent):

{
  "jsonrpc": "2.0",
  "method": "invoke_agent",
  "params": {
    "agent_id": "text_summarizer_agent",
    "input_data": "The full text of the article to be summarized..."
  },
  "id": "agent-request-456"
}

Example A2A Response:

{
  "jsonrpc": "2.0",
  "result": {
    "summary": "This is the summarized text..."
  },
  "id": "agent-request-456"
}

A Unified Agentic System

graph TD
    User -- "User Request" --> TriageAgent

    subgraph Agent_Ecosystem
        TriageAgent(Triage Agent)
        DataAnalysisAgent(Data Analysis Agent)
    end

    subgraph Tool_Ecosystem
        SlackServer(Slack MCP Server)
        DatabaseServer(Database MCP Server)
    end

    TriageAgent -- "Communicate via ACP (RESTful API)" --> DataAnalysisAgent
    DataAnalysisAgent -- "Communicate via ACP (RESTful API)" --> TriageAgent

    TriageAgent -- "Invoke Tool via MCP (JSON-RPC)" --> SlackServer
    DataAnalysisAgent -- "Invoke Tool via MCP (JSON-RPC)" --> DatabaseServer

Why JSON-RPC is a Good Fit for AI

• Simplicity and Readability: The text-based nature of JSON makes it easy for developers to debug and inspect the communication between agents and tools.
• Language Agnostic: An orchestrator agent written in Python can easily communicate with a tool written in Go or Node.js.
• Statelessness: Each request is self-contained, which simplifies the architecture of both the client and the server.
• Flexibility: JSON-RPC is transport-agnostic, meaning it can be used over HTTP, WebSockets, or any other transport layer, providing flexibility for different use cases.

While it may not have the raw performance of binary protocols like gRPC, its simplicity and flexibility make it an excellent choice for the rapidly evolving world of AI, where ease of development and interoperability are often more important than squeezing out every last drop of performance. As the AI ecosystem continues to mature, we can expect to see JSON-RPC play an increasingly important role in how AI models and agents communicate.

References

• JSON-RPC 2.0 Specification: https://www.jsonrpc.org/specification
• Model Context Protocol (MCP): https://modelcontextprotocol.io/
• Agent Communication Protocol (ACP): https://agentcommunicationprotocol.dev/
• Agent Network Protocol (ANP): https://github.com/agent-network-protocol/AgentNetworkProtocol