Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I provided a self-contained, minimal, reproducible example that a maintainer can copy and run AS IS, including all necessary imports and data.

Example Code

from functools import partial
from langchain.agents.middleware.summarization import _get_approximate_token_counter
from langchain_core.messages import HumanMessage
from langchain_core.messages.utils import count_tokens_approximately
from langchain_google_vertexai.model_garden import ChatAnthropicVertex

model = ChatAnthropicVertex(
    model_name="claude-haiku-4-5",
    project="your-project",
    location="us-east5",
    max_output_tokens=256,
    profile={"max_input_tokens": 200_000},
)

# The bug: _get_approximate_token_counter checks model._llm_type == "anthropic-chat"
# but ChatAnthropicVertex returns "anthropic-chat-vertexai"
print(f"model._llm_type: {model._llm_type!r}")  # 'anthropic-chat-vertexai'
print(f"Match: {model._llm_type == 'anthropic-chat'}")  # False

# Result: counter uses default 4.0 chars/token instead of 3.3
counter = _get_approximate_token_counter(model)
correct = partial(count_tokens_approximately, chars_per_token=3.3)

msg = [HumanMessage(content="x" * 600_000)]
print(f"Middleware estimate: {counter(msg)}")   # ~150K
print(f"Correct estimate:   {correct(msg)}")    # ~182K

Full standalone reproduction script with API calls demonstrating the rejection: https://gist.github.com/Jordanh1996/af56156da6cfab82917215dd340f76ac

Error Message and Stack Trace (if applicable)

BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'prompt is too long: 202018 tokens > 200000 maximum'}}

Description

_get_approximate_token_counter in langchain.agents.middleware.summarization uses an exact match model._llm_type == "anthropic-chat" to detect Anthropic models and apply the correct chars_per_token=3.3. But ChatAnthropicVertex (Claude via Vertex AI) returns _llm_type = "anthropic-chat-vertexai", so it falls through to the default 4.0 chars/token.

This underestimates token count by ~16%. When used with SummarizationMiddleware, the trigger threshold (85% of 200K = 170K) is never reached according to the estimate, but the actual prompt is already past 200K. The API rejects it.

Two additional safety mechanisms are also inactive for Vertex AI:

• use_usage_metadata_scaling in count_tokens_approximately requires response_metadata["model_provider"] to be set on AI messages. ChatAnthropicVertex never sets this (unlike ChatAnthropic which sets "anthropic"). This is a langchain-google-vertexai issue.
• _should_summarize_based_on_reported_tokens has the same dependency and is equally inactive.

Suggested fix: Change the check to model._llm_type.startswith("anthropic-chat") to cover both ChatAnthropic and ChatAnthropicVertex.

System Info

langchain: 1.2.10
langchain_core: 1.2.16
langchain_google_vertexai: 3.2.2
langchain_anthropic: 1.3.4
Python: 3.11.8
OS: Darwin (macOS ARM64)