One Agent, Three Frameworks: What Agent Framework, LangGraph and ADK Do When the Model Misbehaves

Every agent framework demo shows the same thing: the model picks the right tool, passes the right arguments, reads the result and answers. Under those conditions the frameworks are interchangeable. They stop being interchangeable when the model does something else: calls a tool that does not exist, sends text where a number belongs, asks for the same refund twice, or never stops. What happens next is decided by defaults that few teams read, and the defaults differ.

This article builds one small agent three ways, with Microsoft Agent Framework, LangGraph and Google’s Agent Development Kit, and gives all of them the same two tools. Then it replaces the model with a script. The script first plays a model that behaves. After that it plays out twelve things that go wrong, in the model, in a tool or at the endpoint, and the test records what each framework does. Earlier articles here introduced the frameworks one at a time: Agent Framework, ADK and LangGraph. This one puts them side by side under the conditions an architect has to plan for.

The experiment

The tools are plain Python functions with type hints and a docstring, the form all three frameworks accept. One reads; the other has a side effect, which is what makes the failure cases matter. Every execution is appended to a ledger, so the test can tell what ran, with which arguments, and when. The lookup fails with a message that names an internal database host, to see where that text ends up.

def lookup_order(order_id: str) -> dict:
    """Looks up one order by its identifier and returns its status and total.

    Args:
        order_id: The order identifier, for example A-100.
    """
    entry = _record("lookup_order", order_id=order_id)
    time.sleep(DELAY)
    entry["finished"] = time.perf_counter()
    if order_id not in ORDERS:
        raise LookupError(f"order {order_id} not found in orders-db-internal-7.corp.example:5432")
    return {"order_id": order_id, **ORDERS[order_id]}


def refund_order(order_id: str, amount: float) -> dict:
    """Refunds part or all of an order to the customer. This moves money.

    Args:
        order_id: The order identifier, for example A-100.
        amount: The amount to refund, in euro.
    """
    entry = _record("refund_order", order_id=order_id, amount=amount)
    entry["finished"] = time.perf_counter()
    return {"order_id": order_id, "refunded": amount, "argument_type": type(amount).__name__}

Each framework needs only a few lines to turn those into an agent. Agent Framework, through its Chat Completions client:

    client = OpenAIChatCompletionClient("stand-in", api_key="not-a-real-key", base_url=openai_url, **settings)
    agent = Agent(client, INSTRUCTIONS, name="support", tools=list(tools))

LangGraph, through create_agent, the builder that replaced the deprecated create_react_agent:

    model = ChatOpenAI(model="stand-in", api_key="not-a-real-key", base_url=openai_url)
    middleware = [ModelCallLimitMiddleware(run_limit=MAX_MODEL_CALLS), ToolErrorMiddleware(lambda *_, **__: TOOL_FAILED)] if hardened else []
    agent = create_agent(model, list(tools), system_prompt=INSTRUCTIONS, middleware=middleware)

ADK, with either model class:

    agent = LlmAgent(name="support", model=model, instruction=INSTRUCTIONS, tools=list(tools), **settings)
    runner = InMemoryRunner(agent=agent, app_name="lab")

ADK is run twice because it has two routes to a model and they share less code than one might expect. Its Gemini class uses Google’s own client and the Gemini API. Its LiteLLM adapter is how it reaches OpenAI-compatible endpoints, including Azure OpenAI.

A model that plays a script

The stand-in is a small HTTP server that answers both OpenAI Chat Completions and the Gemini API. A scenario is a list: what the model returns to the first request, to the second, and so on. A step is a set of tool calls, a piece of text, or an HTTP error.

    {"name": "1. The model behaves", "ask": "Refund 25 euro on order A-100.",
     "script": [{"calls": [LOOKUP_A]}, {"calls": [REFUND_A]}, {"text": "Refunded 25.00 on A-100."}]},
    {"name": "2. Two tool calls in one turn", "ask": "What is the status of A-100 and B-200?",
     "script": [{"calls": [LOOKUP_A, ("lookup_order", {"order_id": "B-200"})]}, {"text": "A-100 delivered, B-200 shipped."}]},
    {"name": "2b. The same two calls, with the tool written as an async function", "ask": "What is the status of A-100 and B-200?", "tools": "async",
     "script": [{"calls": [("lookup_order_async", {"order_id": "A-100"}), ("lookup_order_async", {"order_id": "B-200"})]}, {"text": "A-100 delivered, B-200 shipped."}]},
    {"name": "3. The same refund asked for twice in one turn", "ask": "Refund 25 euro on order A-100.",
     "script": [{"calls": [REFUND_A, REFUND_A]}, {"text": "Refunded."}]},
    {"name": "4. A tool raises an exception", "ask": "What is the status of order Z-999?",
     "script": [{"calls": [("lookup_order", {"order_id": "Z-999"})]}, {"text": "I could not find that order."}]},
    {"name": "5. The model calls a tool that does not exist", "ask": "Cancel order A-100.",
     "script": [{"calls": [("cancel_order", {"order_id": "A-100"})]}, {"text": "I cannot cancel orders."}]},
    {"name": "6. A required argument is missing", "ask": "Refund order A-100.",
     "script": [{"calls": [("refund_order", {"order_id": "A-100"})]}, {"text": "How much should I refund?"}]},
    {"name": "7. An argument has the wrong type", "ask": "Refund order A-100.",
     "script": [{"calls": [("refund_order", {"order_id": "A-100", "amount": "twenty-five"})]}, {"text": "The amount must be a number."}]},

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','signalColor':'#0073BB','signalTextColor':'#0F172A','actorBkg':'#FFF3E0','actorBorder':'#FF9900','actorTextColor':'#232F3E','noteBkgColor':'#FFF8E7','noteBorderColor':'#FF9900','noteTextColor':'#232F3E'}}}%%
sequenceDiagram
    participant A as Agent framework
    participant S as Scripted stand-in
    participant T as Tool function
    A->>S: Request 1 with instructions, tools and the user message
    S-->>A: Step 1 of the script, for example a tool call
    A->>T: Run the tool, or refuse to
    T-->>A: Result or exception
    A->>S: Request 2 with the same, plus what it tells the model
    S-->>A: Step 2 of the script

Three things are recorded for every run: whether the agent answered or raised, which tools actually executed, and what the framework told the model about them in its next request. That last one is read from the wire, not from the framework’s own logs.

When the model behaves

With a well-behaved script, a lookup, a refund and a closing sentence, all four produce the same answer from the same two executions in three model calls. The requests differ only in small ways.

== What each framework sent on the first request of scenario 1

Agent Framework: /v1/chat/completions, 1,132 bytes, user agent agent-framework-python/1.26.0
  instructions sent as a 'system' message; other request fields: {"stream": false, "tool_choice": "auto"}
  refund_order as declared to the model: {"description": "Refunds part or all of an order to the customer. This moves money.\n\nArgs:\n    order_id: The order identifier, for example A-100.\n    amount: The amount to refund, in euro.\n", "name": "refund_order", "parameters": {"properties": {"amount": {"title": "Amount", "type": "number"}, "order_id": {"title": "Order Id", "type": "string"}}, "required": ["order_id", "amount"], "title": "refund_order_input", "type": "object"}}
  a tool result as sent back: {"order_id": "A-100", "status": "delivered", "total": 80.0}

LangGraph: /v1/chat/completions, 982 bytes, user agent AsyncOpenAI/Python 3.51.0
  instructions sent as a 'system' message; other request fields: {"stream": false}
  refund_order as declared to the model: {"description": "Refunds part or all of an order to the customer. This moves money.\n\nArgs:\n    order_id: The order identifier, for example A-100.\n    amount: The amount to refund, in euro.", "name": "refund_order", "parameters": {"properties": {"amount": {"type": "number"}, "order_id": {"type": "string"}}, "required": ["order_id", "amount"], "type": "object"}}
  a tool result as sent back: {"order_id": "A-100", "status": "delivered", "total": 80.0}

ADK (Gemini API): /v1beta/models/gemini-stand-in:generateContent, 1,170 bytes, user agent google-genai-sdk/2.34.0 gl-python/3.13.18 google-adk/2.14.1 gl-pyth...
  instructions sent as the systemInstruction field; other request fields: {"generationConfig": {}}
  refund_order as declared to the model: {"description": "Refunds part or all of an order to the customer. This moves money.\n\nArgs:\n    order_id: The order identifier, for example A-100.\n    amount: The amount to refund, in euro.", "name": "refund_order", "parameters_json_schema": {"properties": {"amount": {"title": "Amount", "type": "number"}, "order_id": {"title": "Order Id", "type": "string"}}, "required": ["order_id", "amount"], "title": "refund_orderParams", "type": "object"}}
  a tool result as sent back: {"order_id": "A-100", "status": "delivered", "total": 80.0}

ADK (LiteLLM): /v1/chat/completions, 1,144 bytes, user agent AsyncOpenAI/Python 3.51.0
  instructions sent as a 'system' message; other request fields: {}
  refund_order as declared to the model: {"description": "Refunds part or all of an order to the customer. This moves money.\n\nArgs:\n    order_id: The order identifier, for example A-100.\n    amount: The amount to refund, in euro.", "name": "refund_order", "parameters": {"properties": {"amount": {"title": "Amount", "type": "number"}, "order_id": {"title": "Order Id", "type": "string"}}, "required": ["order_id", "amount"], "title": "refund_orderParams", "type": "object"}}
  a tool result as sent back: {"order_id": "A-100", "status": "delivered", "total": 80.0}

The same Python function becomes four slightly different tool declarations. All of them send the whole docstring, argument section included, as the description, so the docstring is prompt text and should be written as such. None of them marks the schema as strict, and none sends a sampling temperature. The tool result goes back as JSON in every case. So far, a team could swap one framework for another and see no difference.

Several tool calls in one turn

Models often return more than one tool call in a single response. Two questions follow: do the calls run at the same time, and what happens when two of them are identical?

Agent Framework and LangGraph run ordinary functions concurrently, which means on different threads. A tool that touches shared state, or a client that is not thread-safe, will be called from two threads at once the first time a model batches its calls. ADK runs ordinary functions in sequence and only overlaps them when they are written as coroutines.

No framework notices that two calls in one turn are the same call. The refund ran twice in all four. That is correct behaviour for a framework, which cannot know that a second identical call is a mistake, and it is the strongest argument in this article for a rule that predates agents: a tool that moves money takes an idempotency key and enforces it on the server.

When a tool fails

== 4. A tool raises an exception
   script: [{"calls": [["lookup_order", {"order_id": "Z-999"}]]}, {"text": "I could not find that order."}]
Agent Framework    answered     2 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'I could not find that order.'
                   told the model: Error: Function failed.
LangGraph          raised       1 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'LookupError: order Z-999 not found in orders-db-internal-7.corp.example:5432'
ADK (Gemini API)   raised       1 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'LookupError: order Z-999 not found in orders-db-internal-7.corp.example:5432'
ADK (LiteLLM)      raised       1 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'LookupError: order Z-999 not found in orders-db-internal-7.corp.example:5432'

The split here is the widest in the whole test. Agent Framework catches the exception, tells the model only that the function failed, and carries on; the database host in the message goes nowhere. LangGraph and ADK let the exception leave the agent. The run ends, the conversation turn is lost, and the caller gets the raw message. Neither is wrong. One treats a tool failure as something the model should handle, the other as something the application should. But a team moving from one to the other inherits the opposite policy without changing a line.

A call to a tool that does not exist is handled the same way by all four: nothing runs, and the model is told. LangGraph and ADK add the list of valid tool names to the message, which gives the model a chance to correct itself.

When the arguments are wrong

== 6. A required argument is missing
   script: [{"calls": [["refund_order", {"order_id": "A-100"}]]}, {"text": "How much should I refund?"}]
Agent Framework    answered     2 model calls     0.1 s  tools run: none
                   result: 'How much should I refund?'
                   told the model: Error: Argument parsing failed.
LangGraph          answered     2 model calls     0.1 s  tools run: none
                   result: 'How much should I refund?'
                   told the model: Error invoking tool 'refund_order' with kwargs {'order_id': 'A-100'} with error: amount: Field required Please fix the error and try again.
ADK (Gemini API)   answered     2 model calls     0.1 s  tools run: none
                   result: 'How much should I refund?'
                   told the model: {"error": "Invoking `refund_order()` failed as the following mandatory input parameters are not present:\namount\nYou could retry calling this tool, but it is IMPORTAN...
ADK (LiteLLM)      answered     2 model calls     0.1 s  tools run: none
                   result: 'How much should I refund?'
                   told the model: {"error": "Invoking `refund_order()` failed as the following mandatory input parameters are not present:\namount\nYou could retry calling this tool, but it is IMPORTAN...

== 7. An argument has the wrong type
   script: [{"calls": [["refund_order", {"order_id": "A-100", "amount": "twenty-five"}]]}, {"text": "The amount must be a number."}]
Agent Framework    answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: Error: Argument parsing failed.
LangGraph          answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: Error invoking tool 'refund_order' with kwargs {'order_id': 'A-100', 'amount': 'twenty-five'} with error: amount: Input should be a valid number, unable to parse strin...
ADK (Gemini API)   answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount='twenty-five')
                   result: 'The amount must be a number.'
                   told the model: {"order_id": "A-100", "refunded": "twenty-five", "argument_type": "str"}
ADK (LiteLLM)      answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount='twenty-five')
                   result: 'The amount must be a number.'
                   told the model: {"order_id": "A-100", "refunded": "twenty-five", "argument_type": "str"}

== 8. A number arrives as text
   script: [{"calls": [["refund_order", {"order_id": "A-100", "amount": "25"}]]}, {"text": "Refunded."}]
Agent Framework    answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}
LangGraph          answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}
ADK (Gemini API)   answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount='25')
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": "25", "argument_type": "str"}
ADK (LiteLLM)      answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount='25')
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": "25", "argument_type": "str"}

A missing argument is caught everywhere. A wrong type is not. Agent Framework and LangGraph validate arguments against the function’s type hints, refuse the call, and coerce a number that arrives as text. ADK passes the arguments through as they came: refund_order was executed with the string 'twenty-five' as its amount, and with '25' as a string where the function expects a float. In Python nothing stops that at the call. What happens next depends on the first line of the function that does arithmetic.

== 9. The arguments are not valid JSON
   script: [{"calls": [["refund_order", "{\"order_id\": \"A-100\", \"amount\": 25"]]}, {"text": "Something went wrong."}]
Agent Framework    answered     2 model calls     0.1 s  tools run: none
                   result: 'Something went wrong.'
                   told the model: Error: Argument parsing failed.
LangGraph          answered     1 model calls     0.0 s  tools run: none
                   result: ''
ADK (Gemini API)   not applicable: the Gemini API carries arguments as an object, so they cannot be malformed JSON
ADK (LiteLLM)      answered     1 model calls     0.0 s  tools run: none
                   result: ''

The last case is specific to Chat Completions, where arguments travel as a JSON string that the model can truncate or garble. Agent Framework tells the model the arguments could not be parsed. LangGraph and ADK’s LiteLLM route end the run after one model call with an empty answer and no error. The user sees nothing, the logs show a successful run, and the tool was never called. An empty final answer is worth an alert of its own.

When the model endpoint fails

== 11. Two 429 responses, then an answer
   script: [{"status": 429, "message": "Rate limit reached"}, {"status": 429, "message": "Rate limit reached"}, {"text": "Hello."}]
Agent Framework    answered     3 model calls     1.2 s  tools run: none
                   result: 'Hello.'
LangGraph          answered     3 model calls     1.3 s  tools run: none
                   result: 'Hello.'
ADK (Gemini API)   raised       1 model calls     0.0 s  tools run: none
                   result: '_ResourceExhaustedError: On how to mitigate this issue, please refer to: https://google.github.io/adk-docs/agents/models/google-gemini/#error-code-429-resource_exhaust...'
ADK (LiteLLM)      answered     3 model calls     1.3 s  tools run: none
                   result: 'Hello.'

== 12. Two 500 responses, then an answer
   script: [{"status": 500, "message": "Internal error"}, {"status": 500, "message": "Internal error"}, {"text": "Hello."}]
Agent Framework    answered     3 model calls     1.2 s  tools run: none
                   result: 'Hello.'
LangGraph          answered     3 model calls     1.2 s  tools run: none
                   result: 'Hello.'
ADK (Gemini API)   raised       1 model calls     0.0 s  tools run: none
                   result: "ServerError: 500 UNAVAILABLE. {'error': {'message': 'Internal error', 'type': 'server_error', 'code': 500, 'status': 'UNAVAILABLE'}}"
ADK (LiteLLM)      answered     3 model calls     1.2 s  tools run: none
                   result: 'Hello.'

Three of the four recover from two rate-limit or server errors, and none of that is the frameworks’ doing: all three sit on the OpenAI Python client, which retries twice by default. Asked for ever, each gives up after three attempts in about a second and a half. ADK’s Gemini route uses Google’s client, which has no retries unless they are configured, so one 429 ends the run. A fleet of agents all retrying on the same schedule is its own problem, but zero retries against a hosted model is not a production setting.

When the model never stops

== 14. The model never stops calling a tool
   script: [{"calls": [["lookup_order", {"order_id": "A-100"}]], "repeat": true}]
Agent Framework    answered    41 model calls     2.0 s  tools run: 40 executions
                   result: 'Function invocation limit reached before a final answer could be produced.'
LangGraph          cut off    601 model calls    67.7 s  tools run: 600 executions
                   result: 'still going after 600 model calls'
ADK (Gemini API)   raised     500 model calls    87.9 s  tools run: 500 executions
                   result: 'LlmCallsLimitExceededError: Max number of llm calls limit of `500` exceeded'
ADK (LiteLLM)      raised     500 model calls    37.3 s  tools run: 500 executions
                   result: 'LlmCallsLimitExceededError: Max number of llm calls limit of `500` exceeded'

defaults read from the installed packages: Agent Framework max_iterations 40, max_consecutive_errors_per_request 3; LangGraph recursion limit 10007 graph steps; ADK max_llm_calls 500; openai client max_retries 2

Agent Framework stops at 40 model round trips and returns a sentence saying so, as if it were the answer. ADK stops at 500 model calls with an exception. LangGraph was still going when the test cut it off at 600; its limit in this release is 10,007 graph steps, about five thousand model calls, where earlier releases defaulted to 25. A loop like this is rare with a capable model and common with a confused one, and every iteration is a billed request with a growing context. The default is a budget decision that someone else made.

Making the four agree

Each difference above has a setting. The sample builds every agent a second time with four policies stated explicitly: at most eight model calls, tool failures reported to the model without their detail, arguments checked against the function’s types, and retries on the client that has none.

Agent Framework already hides tool errors and validates arguments, so it needs one line:

    settings = {"function_invocation_configuration": {"max_iterations": MAX_MODEL_CALLS}} if hardened else {}
    client = OpenAIChatCompletionClient("stand-in", api_key="not-a-real-key", base_url=openai_url, **settings)

LangGraph takes both policies as middleware:

    middleware = [ModelCallLimitMiddleware(run_limit=MAX_MODEL_CALLS), ToolErrorMiddleware(lambda *_, **__: TOOL_FAILED)] if hardened else []

ADK needs the most, because the checks are not there by default: a validating wrapper around each tool, an error callback, a run limit, and retry options for the Gemini client.

    if hardened:
        # ADK passes the model's arguments to the function as they arrive. validate_call checks them first.
        tools = [pydantic.validate_call(tool) for tool in tools]
        settings["on_tool_error_callback"] = lambda tool, arguments, context, error: {"error": TOOL_FAILED}
        run_config = RunConfig(max_llm_calls=MAX_MODEL_CALLS)

    retry = types.HttpRetryOptions(attempts=3, initial_delay=0.2, max_delay=1.0, http_status_codes=[429, 500, 502, 503, 504]) if hardened else None

== The same scenarios with a few lines of configuration: at most 8 model calls, tool failures reported without detail, arguments type-checked, retries on

-- 4. A tool raises an exception
Agent Framework    answered     2 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'I could not find that order.'
                   told the model: Error: Function failed.
LangGraph          answered     2 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'I could not find that order.'
                   told the model: The tool failed. Do not retry; tell the user it could not be done.
ADK (Gemini API)   answered     2 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'I could not find that order.'
                   told the model: {"error": "The tool failed. Do not retry; tell the user it could not be done."}
ADK (LiteLLM)      answered     2 model calls     0.3 s  tools run: lookup_order(order_id='Z-999')
                   result: 'I could not find that order.'
                   told the model: {"error": "The tool failed. Do not retry; tell the user it could not be done."}

-- 7. An argument has the wrong type
Agent Framework    answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: Error: Argument parsing failed.
LangGraph          answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: Error invoking tool 'refund_order' with kwargs {'order_id': 'A-100', 'amount': 'twenty-five'} with error: amount: Input should be a valid number, unable to parse strin...
ADK (Gemini API)   answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: {"error": "The tool failed. Do not retry; tell the user it could not be done."}
ADK (LiteLLM)      answered     2 model calls     0.1 s  tools run: none
                   result: 'The amount must be a number.'
                   told the model: {"error": "The tool failed. Do not retry; tell the user it could not be done."}

-- 8. A number arrives as text
Agent Framework    answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}
LangGraph          answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}
ADK (Gemini API)   answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}
ADK (LiteLLM)      answered     2 model calls     0.1 s  tools run: refund_order(order_id='A-100', amount=25.0)
                   result: 'Refunded.'
                   told the model: {"order_id": "A-100", "refunded": 25.0, "argument_type": "float"}

-- 11. Two 429 responses, then an answer
Agent Framework    answered     3 model calls     1.3 s  tools run: none
                   result: 'Hello.'
LangGraph          answered     3 model calls     1.4 s  tools run: none
                   result: 'Hello.'
ADK (Gemini API)   answered     3 model calls     1.2 s  tools run: none
                   result: 'Hello.'
ADK (LiteLLM)      answered     3 model calls     1.5 s  tools run: none
                   result: 'Hello.'

-- 14. The model never stops calling a tool
Agent Framework    answered     9 model calls     0.4 s  tools run: 8 executions
                   result: 'Function invocation limit reached before a final answer could be produced.'
LangGraph          answered     8 model calls     0.4 s  tools run: 8 executions
                   result: 'Model call limits exceeded: run limit (8/8)'
ADK (Gemini API)   raised       8 model calls     0.4 s  tools run: 8 executions
                   result: 'LlmCallsLimitExceededError: Max number of llm calls limit of `8` exceeded'
ADK (LiteLLM)      raised       8 model calls     0.4 s  tools run: 8 executions
                   result: 'LlmCallsLimitExceededError: Max number of llm calls limit of `8` exceeded'
   model calls when capped at eight: Agent Framework 9, LangGraph 8, ADK (Gemini API) 8, ADK (LiteLLM) 8
ok    hardened: every framework stops within nine model calls when the model never stops
ok    hardened: a failing tool no longer ends the run, and the database host in its message reaches neither the model nor the caller
ok    hardened: no framework runs the refund when the amount is not a number
ok    hardened: all four answer after two 429 responses

With those settings a failing tool no longer ends the run in any of them, the database host reaches neither the model nor the caller, no refund runs with a non-numeric amount, and all four survive two rate-limit responses. Three differences remain, and they are worth knowing.

• The limit ends differently. Agent Framework and LangGraph return a sentence in place of an answer; ADK raises. Code that calls the agent has to recognise both, or a limit message will be shown to a user as the agent’s reply.
• Eight does not mean eight everywhere. Agent Framework’s limit counts round trips that run tools and then makes one more call, so it sent nine requests.
• Validation messages still reach the model in LangGraph. The error middleware handles exceptions raised inside a tool. Argument validation failures are reported separately, with the arguments echoed back. That is useful to the model and harmless here, but it is a second path to review if tool arguments can contain sensitive values.

Security and operations

Cost

The cost of these defaults is the cost of the worst run, not the average one. Take a conversation whose requests average 20,000 input tokens, on a model priced at $2 per million input tokens, and ignore caching and output. A loop that runs to ADK’s default of 500 calls is 10 million input tokens, or $20, for one stuck conversation. At Agent Framework’s 40 it is $1.60. At a limit of eight it is 32 cents. These are illustrative figures from stated assumptions, not measurements; the point is the ratio, and that the limit is the only thing between a confused model and the larger number. Retries multiply the same way: three attempts per call is up to three times the requests during an incident, when the provider is already struggling.

Decision framework

Nothing in this test ranks the frameworks. Each default is defensible, and each can be changed. The choice of framework should follow the rest of the estate; the settings should not be left to it.

Key takeaways

• With a well-behaved model, Agent Framework, LangGraph and ADK gave the same answer from the same tool calls. With a misbehaving one, they agreed on two of eight scenarios.
• A tool exception was hidden from the model and survived in Agent Framework, and ended the run in LangGraph and ADK, carrying an internal host name out with it.
• ADK executed a refund with the text ‘twenty-five’ as its amount. Agent Framework and LangGraph rejected it. Type hints are not validation unless the framework makes them so.
• A model that never stops was allowed 40 calls by Agent Framework, 500 by ADK and about 5,000 by LangGraph, whose default was 25 in earlier releases. Set the limit yourself.
• A few lines of configuration per framework brought the four into line. Keep a scripted model in the pipeline so that the next change of default shows up as a failed test.

If you run one of these frameworks in production, I would like to know which default surprised you first. Leave a comment.

Glossary

• Agent framework— A library that runs a model in a loop with tools: it sends the conversation, executes the tool calls the model asks for, and sends the results back.
• Tool call— A structured request from the model to run a named function with given arguments.
• Chat Completions— OpenAI’s widely copied request format, in which tool arguments travel as a JSON string.
• Gemini API— Google’s request format, in which tool arguments travel as a JSON object.
• Stand-in— Here, a local server that answers in a model’s wire format from a script.
• Idempotency key— A value sent with an operation so that the service performs it once however many times the request arrives.
• Recursion limit— LangGraph’s cap on graph steps in one run; a model call and a tool step are two steps.
• Middleware— In LangChain agents and Agent Framework, code that wraps model or tool calls to add policy such as limits and error handling.
• LiteLLM— A library that translates one calling convention to many model providers; ADK uses it for non-Google models.
• Coercion— Converting a value to the declared type, such as the text “25” to the number 25.0.

References & Further Reading

• → microsoft/agent-framework— The framework for .NET and Python
• → LangChain agents: create_agent— The agent builder used here, and its middleware
• → langchain-ai/langgraph— The orchestration layer underneath
• → google/adk-python— Agent Development Kit for Python
• → Gemini API: function calling— How tool calls and arguments are represented
• → LiteLLM: OpenAI-compatible endpoints— The route ADK takes to Chat Completions
• → Pydantic: validate_call— The decorator used to check tool arguments in ADK
• → Sample code for this article— Tools, agent builders, stand-in, scenarios and results