Skip to content

Black Friday: up to 61% off our packs until November 30.

See the packs

LangSmith Tool Gateway Setup Guide: Credentials, Tool Kits, Policies and Approvals for Agent Tool Calls

LangSmith Tool Gateway entered public beta on November 19, 2026. It moves credentials, identity and access for agent tool calls into one managed layer. This guide uses a See / Act As / Allow model to walk through registering MCP servers, building tool kits, choosing between per-user and service-key identity, and adding block and approval policies. It also covers the documented limits and where these controls stop protecting you.

What LangSmith Tool Gateway Is and Why It Exists

LangSmith Tool Gateway is a managed tool layer that handles credentials, identity and access for every agent tool call, in LangChain's own description. It sits between your agents and the MCP servers they call. Individual agents no longer have to settle those questions themselves.

LangChain launched Tool Gateway in public beta on November 19, 2026, as Unite.AI reported. The beta is available on LangSmith Plus and Enterprise plans. On LangSmith Cloud it covers the GCP US, GCP EU, GCP APAC and AWS US regions, with Azure US excluded.

Take the large US retailer LangChain describes in its launch post: its engineering team has built over 50 MCP servers exposing more than a thousand tools. Any single agent should touch only a small, approved slice of that catalog, with credentials that fit its context. That kind of sprawl, with tools and credentials spread across many servers and agents, is what the gateway is meant to bring under one set of controls.

One note before the details. Beta products change, and several implementation details matter for working code: response field names, configuration settings, endpoint formats, authentication headers, transport requirements and plan coverage for individual features. The examples below use placeholders for these. Confirm each one against the current docs and real responses during testing, and copy the values exactly rather than guessing.

The See / Act As / Allow governance model

Excessive Agency, in the definition used by the OWASP Gen AI Security Project, is the vulnerability that lets an LLM take damaging actions in response to unexpected, ambiguous or manipulated outputs. Its three typical root causes are excessive functionality, excessive permissions and excessive autonomy.

We map each of those root causes to one of the gateway's controls and call the result See / Act As / Allow:

  • See (excessive functionality): which tools does this agent see? Tool kits.
  • Act As (excessive permissions): whose credentials does a call carry? Identity settings.
  • Allow (excessive autonomy): should this specific call run? Block and approval policies.

The layers do not cover for each other. A refunds agent with a two-tool kit still carries whatever its credential allows. If that credential is the backend service key and no policy blocks a destructive tool, the small kit has narrowed the menu without narrowing the risk.

How a Tool Call Flows Through the Gateway

A tool call involves three separate authentication steps, broken down in full on the How Tool Gateway works page. For design purposes two of them matter most. The caller authenticates only to LangSmith. Tool Gateway then picks a stored credential and uses it to authenticate to the MCP server. As a result, the caller never holds the server's token or API key. It is the same pattern as a credential proxy that keeps API keys out of AI agents: the agent asks, the proxy holds the secret.

The design lines up with the MCP specification's authorization guidance. When an MCP server calls upstream APIs, it uses a separate token issued by the upstream authorization server and must not pass through the token it received from the MCP client. Authorization servers should also issue short-lived access tokens to reduce the impact of leaked tokens.

When a user has not connected an account

If a call needs a user's credential and that user hasn't connected their account, the call returns a sign-in link. A client that supports MCP URL elicitation shows the link to the user. Other clients get an HTTP 428 response with the link. Once the user connects, later calls use the stored credential.

We suggest your client handle this response as an expected sign-in step rather than an error. Show the link, wait for the user, and retry once:

resp = call_tool(req)
if resp.status == 428:
    link = sign_in_link_from(resp.body)  # placeholder field name
    show_sign_in(link)
    await user_confirms_sign_in()
    resp = call_tool(req)                # retry once, never loop

What the gateway does not do to responses

Each server's response reaches the caller unchanged. LangChain's advice is to treat tool output as untrusted input to your agent. The gateway is not a content filter. Defences such as instruction and data separation against prompt injection remain your job and belong in the agent design.

Setup Walkthrough: Servers, Kits, Search Mode and a First Request

Screens and API paths may change during the beta. These steps therefore track documented concepts rather than clicks.

  1. Register each MCP server. Add the server so the gateway, not your agent, authenticates to it, and review the identity docs before deciding whose credentials its calls should carry.
  2. Create a kit per agent role. Pick tools from your registered servers, name the kit after the role it serves, and keep it small. The tool kits docs cover creating and refreshing kits.
  3. Use search mode for large catalogs. If a role genuinely needs many tools, search mode avoids loading every tool at once.
  4. Send a first request from your backend. A backend caller sends the service key plus an X-LangSmith-Identity header naming the user the call acts for:
POST <your kit's MCP endpoint, as shown in LangSmith>
<auth header carrying the service key>
X-LangSmith-Identity: <user this call acts for>

<standard MCP tool call payload>

The endpoint and auth lines are placeholders, as flagged in the note at the top. Take the exact values from LangSmith and the current docs.

See: Building Curated Tool Kits Without Mistaking Them for Security

A kit is a curated set of tools drawn from the servers registered with the gateway. The point is focus: an agent that handles refunds has no business seeing a calendar tool or a deployment tool. Smaller kits also help quality, because LangChain notes that when an agent has hundreds or thousands of tools, loading them all at once increases latency and makes choosing the right tool harder. Up to 10,000 tools are allowed per kit, but treat that number as a ceiling rather than a goal.

Kits are not access control

The same caller that uses a kit can also use kits/me/mcp, which serves every tool the caller can use. For that reason LangChain's documentation states plainly that a kit is not an access control. Taking a tool out of a kit does not stop a call the agent's identity is allowed to make. Enforcement has to live in Act As and Allow.

Keep kits fresh

Each server's tool list is cached by Tool Gateway for two hours, per the tool kits docs. Make a kit refresh part of every MCP server release, or accept that agents may see a tool list up to two hours stale after a change. That matters most when a release removes a tool you no longer want agents to reach.

Act As: Choosing Identity for Users and Autonomous Agents

Identity decides whose permissions a tool call carries. In our See / Act As / Allow mapping, this is the layer that addresses excessive permissions. Before choosing a pattern, we suggest reading the identity docs alongside the two approaches below.

Per-user credentials

Each user connects their own account through the sign-in flow above. This suits assistants acting for a person, since the agent can do only what that person can already do upstream.

Service key with the X-LangSmith-Identity header

Backend systems use a service key together with an X-LangSmith-Identity header. Tool Gateway trusts the value in that header as sent. Any caller holding the service key can therefore act as any user. Send the header only from a backend you control and never expose the service key to a browser or an end user; that is LangChain's own guidance. Treat this key like a root credential.

The comparison below is our own assessment, not LangChain documentation.

QuestionPer-user credentialsService key on a backend
Who the call acts forA person who connected their own accountWhatever identity your backend sends in the header
Typical fitInteractive assistants and copilotsBackend services and unattended or scheduled agents
Main riskConnected accounts with broader rights than the task needsA leaked key allows impersonation of any user
Approval policiesWorkable, because the person is present to confirmOnly useful if a person is present; otherwise use block policies

Our rule of thumb: use per-user credentials when a person is present and the action is theirs, and the service key, sent only from a backend you control, when a job runs unattended or acts for the system.

Allow: Writing Block and Approval Policies That Actually Hold

Policies are the enforcement layer, and two defaults in the policies docs shape everything else. A call that no policy matches runs. If a call matches both a block policy and an approval policy, it is blocked. The first default is the one that catches people out: a tool nobody wrote a policy for is a tool every permitted caller can use.

Who actually approves

Under an approval policy, the caller approves the call, not a separate reviewer. Each call is confirmed before it runs, but nothing is routed to an admin. An agent running with no one present cannot approve calls. For that case LangChain recommends a block policy to stop the agent from calling the tool.

Human-in-the-loop control, with a human approving high-impact actions before they happen, is among OWASP's recommended mitigations. Caller approval fits when a user lets an assistant send an email from their own account. It does not fit actions that need a second pair of eyes, such as a refund above a threshold your finance team sets. For those, block the tool at the gateway or build a preflight approval step in your own application.

Capacity

An organisation can have up to 1,000 policies that apply to every workspace, plus up to 100 policies per workspace. Creating a policy past either limit returns a 422 error. Put rules such as blocking irreversible tools for unattended agents at org level and team-specific rules in workspaces. If you generate policies automatically, handle the 422 so a script fails loudly rather than leaving a rule unwritten.

A Rollout Playbook: From Scattered Secrets to One Governed Gateway

Run these steps from lowest risk to highest.

  1. Inventory. List every MCP server and tool your agents use, and tag each by impact: read, write, irreversible or external-facing.
  2. Register servers. Move upstream credentials into the gateway, then confirm no agent configuration still holds an upstream key.
  3. See. Build one kit per agent role with the smallest useful tool set.
  4. Act As. Per-user credentials for assistants acting for people; the service key, from a backend only, for autonomous jobs.
  5. Allow. Block policies first, for unattended agents and irreversible tools, then approval policies for sensitive interactive calls.
  6. Confirm traces are arriving. Check the tool-gateway tracing project and the Usage tab after your first test calls.
  7. Load-test. Test against the documented limits (see Caveats below) under realistic traffic.

Pre-production checklist

  • [ ] No upstream keys or tokens remain in agent code, prompts or client configuration
  • [ ] No unattended agent depends on an approval policy
  • [ ] Clients handle HTTP 428 sign-in responses without looping
  • [ ] Agents treat tool output as untrusted input
  • [ ] Kit refresh is part of the MCP server release process

Observability: What Traces and the Usage Tab Record

Tool call traces record each call's tool, server, user, outcome and duration. Data sent to or returned by the tool is not stored. Sensitive payloads stay out of the record, and so does the detail you might later want for an investigation. Traces land in a tracing project named tool-gateway, which the gateway creates when it records the first call, as the traces docs explain. Workspace traces are free and do not count toward your trace limits. Cost is not a reason to skip them. For quicker checks, the Usage tab lists the 100 most recent tool calls from the last 7 days, with each call's tool, server, outcome, latency and start time. That view suits live rollout checks rather than long-term audit.

If your compliance or incident response process needs payload-level forensics, that logging has to live in your agent or application layer, for example alongside OpenTelemetry GenAI tracing for agents.

Caveats, Beta Limits and When Per-Agent Secrets Are Still Fine

The hard limits listed in LangChain's Tool Gateway documentation, starting from the How Tool Gateway works overview, are a 1 MB request size, an 8 MB response size, 60 seconds for the MCP server to start responding, and 100 requests per second with bursts up to 1,000. Tools that return large documents or depend on slow upstream systems are the ones to load-test first.

During the beta, access is limited to Plus and Enterprise plans and the four listed cloud regions. Beta behaviour can change, and this guide reflects the documentation available at launch. We suggest settling the implementation details flagged at the top, such as transport requirements and plan coverage for governance features, before you plan a migration.

When a gateway wins

A gateway pays off when you have many MCP servers and many agents, when users each need delegated access, or when a security team wants one place to define and review policy.

When per-agent secrets may be enough

Not every team needs this. If you run one agent with a handful of tools, a single owner and no user-delegated access, a central gateway may add more process than protection.

Key Takeaways

  1. LangSmith Tool Gateway entered public beta on November 19, 2026, on Plus and Enterprise plans, in GCP US, GCP EU, GCP APAC and AWS US. Azure US is excluded.
  2. The See / Act As / Allow model maps OWASP's Excessive Agency root causes to kits, identity and policies. Each layer has its own limits.
  3. The caller authenticates only to LangSmith and never holds upstream MCP server tokens or API keys, consistent with the MCP specification's ban on token passthrough.
  4. Tool kits shape what an agent sees but are not access control, because a caller can still reach kits/me/mcp and every tool their identity allows.
  5. With the X-LangSmith-Identity header, anyone holding the service key can act as any user. Keep that key on a backend you control.
  6. Policies allow calls by default, block wins over approval, and the caller is the approver. Unattended agents need block policies.
  7. Traces leave out payloads and tool output passes through unchanged. Plan around the 1 MB request, 8 MB response, 60 second and 100 requests per second limits.

Conclusion

Start with the inventory. Tag every MCP tool your agents use as read, write, irreversible or external-facing, and write block policies for the irreversible and external-facing ones before any agent moves behind the gateway. Then run a single agent role through end to end, with its own small kit and the identity mode that matches who is actually present. Widen the rollout only after its calls show up in the tool-gateway tracing project and load tests stay inside the documented limits.

The exception is the small team. If you run one agent with a handful of tools and one owner, a narrowly scoped server-side secret with short-lived tokens may still be the simpler choice. The gateway can wait until a second agent or delegated user access arrives.

Frequently Asked Questions

Is LangSmith Tool Gateway generally available, and which plans and regions support it?

Not yet. It launched in public beta on November 19, 2026. The beta is available on LangSmith Plus and Enterprise plans, and on LangSmith Cloud it covers GCP US, GCP EU, GCP APAC and AWS US. Azure US is excluded. Check the current docs before committing, since beta scope can change.

Do tool kits stop an agent from calling tools outside the kit?

No. LangChain's docs state that a kit is not an access control, because the same caller can use kits/me/mcp, which serves every tool the caller can use. Kits shape what the agent sees. To restrict what it can actually do, rely on identity scoping and block policies.

What should my client do when it receives an HTTP 428?

Treat it as a sign-in step. The 428 response carries a sign-in link for a user who has not connected their account. Show the link, wait for the user to confirm sign-in, then retry the original call once. To find the field that holds the link, check the current docs or inspect a real response during testing.

Who approves a call under an approval policy, and what should I do for unattended agents?

The caller approves the call. There is no separate reviewer or admin queue. An agent running with no one present cannot approve calls, so LangChain recommends block policies to stop such agents from using a tool. If you need a second person to review high-impact actions, build that step into your own application or block the tool.

Does Tool Gateway store the inputs and outputs of tool calls?

No. Tool call traces record each call's tool, server, user, outcome and duration, but data sent to or returned by the tool is not stored. If you need payload-level records for compliance or forensics, plan that logging in your own application layer.

Does the gateway sanitise tool responses against prompt injection?

No. Tool Gateway passes each server's response to the caller unchanged, and the docs advise treating tool output as untrusted input to your agent. Prompt injection defences still belong in your agent design.

What are the documented size, timeout and rate limits?

The documented hard limits are a 1 MB request size, an 8 MB response size, 60 seconds for the MCP server to start responding, and 100 requests per second with bursts up to 1,000. Load-test tools that return large documents or depend on slow upstream systems before going to production.

Sources

  1. https://www.langchain.com/blog/langsmith-tool-gateway
  2. https://docs.langchain.com/langsmith/tool-gateway-how-it-works
  3. https://docs.langchain.com/langsmith/tool-gateway-quickstart
  4. https://docs.langchain.com/langsmith/tool-gateway-tool-kits
  5. https://docs.langchain.com/langsmith/tool-gateway-identity
  6. https://docs.langchain.com/langsmith/tool-gateway-policies
  7. https://docs.langchain.com/langsmith/tool-gateway-traces
  8. https://www.unite.ai/langchain-launches-langsmith-tool-gateway-to-govern-agent-tool-calls/
  9. https://superpowerdaily.com/posts/langchain-launches-a-gateway-to-control-which-tools-ai-agents-can-use
  10. https://modelcontextprotocol.io/specification/2025-03-26/basic/transports
  11. https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization/security-considerations
  12. https://genai.owasp.org/llmrisk/llm062025-excessive-agency/

Share this article

Hamza Diaz

Written by

Hamza Diaz

Hamza Diaz is the founder of Optijara, where he builds practical AI agents, automation systems, and Copilot workflows for service businesses. He writes about AI operations, agent strategy, and real-world implementation for teams that want usable systems instead of hype.

All articles

Next step

Have something to build?

Tell us what you need. We reply with a clear scope and a fixed quote.