diff --git a/.lycheeignore b/.lycheeignore index 72b96df258..66396b3f9c 100644 --- a/.lycheeignore +++ b/.lycheeignore @@ -5,7 +5,7 @@ # Google Cloud Console (requires authentication, returns HTTP/2 errors) ^https://console\.(cloud\.google|developers\.google)\.com -# Placeholder URL with user-specific path +# URL contains bracket placeholder (e.g. /s/[project-id]) ^https://app\.phoenix\.arize\.com/s/\[ # Requires authentication diff --git a/docs/agents/index.md b/docs/agents/index.md index 41e9bf2e31..dff532e314 100644 --- a/docs/agents/index.md +++ b/docs/agents/index.md @@ -86,4 +86,5 @@ Now that you have an overview of the different agent types available in ADK, div * [**Workflow Agents:**](workflow-agents/index.md) Learn how to orchestrate tasks using `SequentialAgent`, `ParallelAgent`, and `LoopAgent` for structured and predictable processes. * [**Custom Agents:**](custom-agents.md) Discover the principles of extending `BaseAgent` to build agents with unique logic and integrations tailored to your specific needs. * [**Multi-Agents:**](multi-agents.md) Understand how to combine different agent types to create sophisticated, collaborative systems capable of tackling complex problems. +* [**Agent Routing:**](routing.md) Dynamically select between multiple agents at runtime using router functions for fallback, A/B testing, and auto-routing. * [**Models:**](/agents/models/) Learn about the different LLM integrations available and how to select the right model for your agents. diff --git a/docs/agents/models/index.md b/docs/agents/models/index.md index 0b5906280e..2858bffa6f 100644 --- a/docs/agents/models/index.md +++ b/docs/agents/models/index.md @@ -9,25 +9,30 @@ integrate various Large Language Models (LLMs) into your agents. This section details how to leverage Gemini and integrate other popular models effectively, including those hosted externally or running locally. -ADK primarily uses two mechanisms for model integration: +ADK provides several mechanisms for model integration: -1. **Direct String / Registry:** For models tightly integrated with Google Cloud, - such as Gemini models accessed via Google AI Studio or Agent Platform, or models - hosted on Agent Platform endpoints. You access these models by providing the model name or endpoint resource string and ADK's internal registry - resolves this string to the appropriate backend client. +1. **Direct String / Registry:** For models tightly integrated with Google + Cloud, such as Gemini models accessed via Google AI Studio or Agent Platform, + or models hosted on Agent Platform endpoints. You access these models by + providing the model name or endpoint resource string and ADK's internal + registry resolves this string to the appropriate backend client. * [Gemini models](/agents/models/google-gemini/) * [Claude models](/agents/models/anthropic/) * [Agent Platform hosted models](/agents/models/agent-platform/) -2. **Model connectors:** For broader compatibility, especially models - outside the Google ecosystem or those requiring specific client - configurations, such as models accessed via Apigee or LiteLLM. You instantiate a specific wrapper class, such as `ApigeeLlm` or - `LiteLlm`, and pass this object as the `model` parameter - to your `LlmAgent`. +2. **Model connectors:** For broader compatibility, especially models outside + the Google ecosystem or those requiring specific client configurations, such + as models accessed via Apigee or LiteLLM. You instantiate a specific wrapper + class, such as `ApigeeLlm` or `LiteLlm`, and pass this object as the `model` + parameter to your `LlmAgent`. * [Apigee models](/agents/models/apigee/) * [LiteLLM models](/agents/models/litellm/) * [Ollama model hosting](/agents/models/ollama/) * [vLLM model hosting](/agents/models/vllm/) * [LiteRT-LM model hosting](/agents/models/litert-lm/) + +3. **[Model routing](/agents/models/routing/):** For dynamically selecting + between multiple models at runtime using a router function, with automatic + failover on error. diff --git a/docs/agents/models/routing.md b/docs/agents/models/routing.md new file mode 100644 index 0000000000..4c51847bf6 --- /dev/null +++ b/docs/agents/models/routing.md @@ -0,0 +1,64 @@ +# Route between models + +
+ Supported in ADKTypeScript v1.0.0Experimental +
+ +!!! example "Experimental" + + Model routing is experimental and may change in future releases. We welcome + your + [feedback](https://github.com/google/adk-js/issues/new?template=feature_request.md)! + +An `LlmAgent` uses a single model by default. When you need to dynamically +select between different models for each request, you can define a routing +function that chooses which model to use. `RoutedLlm` provides this capability, +enabling model fallback on error, A/B testing between models, and auto-routing +by input complexity. If the selected model fails before producing any output, +the routing function is called again with error context so it can select a +different model. + +Pass a `RoutedLlm` as an `LlmAgent`'s `model` parameter. Use `RoutedLlm` when +only the model varies between routes. If you also need to switch instructions, +tools, or sub-agents, use [`RoutedAgent`](../routing.md) instead. + +## How routing works + +The `LlmRouter` function receives the map of available models and the current +`LlmRequest`, and returns the key of the model to use: + +=== "TypeScript" + + ```typescript + type LlmRouter = ( + models: Readonly>, + request: LlmRequest, + errorContext?: { failedKeys: ReadonlySet; lastError: unknown }, + ) => Promise | string | undefined; + ``` + +The `models` parameter accepts either a `Record` with explicit +keys, or an array of `BaseLlm` instances. If an array is provided, each model's +name is used as its key. + +Failover follows the same rules as +[`RoutedAgent`](../routing.md#how-routing-works): the router is re-called with +`errorContext` only if the selected model fails before yielding any response. +After yielding, errors propagate without retry. The router can return +`undefined` to stop retrying and propagate the last error. + +**Live connections:** `RoutedLlm.connect()` selects the model at connection +time. Once a live connection is established, the model cannot be switched +mid-stream. + +## Basic usage + +The following example creates a `RoutedLlm` that tries a primary model first and +falls back to a secondary model if the primary fails. The router checks +`errorContext.failedKeys` to avoid re-selecting the failed model: + +=== "TypeScript" + + ```typescript + --8<-- "examples/typescript/snippets/agents/models/routing/basic-usage.ts:full" + ``` diff --git a/docs/agents/routing.md b/docs/agents/routing.md new file mode 100644 index 0000000000..67f8e3273c --- /dev/null +++ b/docs/agents/routing.md @@ -0,0 +1,126 @@ +# Route between agents + +
+ Supported in ADKTypeScript v1.0.0Experimental +
+ +!!! example "Experimental" + + Agent routing is experimental and may change in future releases. We welcome + your + [feedback](https://github.com/google/adk-js/issues/new?template=feature_request.md)! + +When building agents for different tasks, you can define a routing function that +selects which one handles each invocation at runtime. `RoutedAgent` provides +this capability, enabling agent fallback on error, A/B testing, planning modes, +and auto-routing by input complexity. If the selected agent fails before +producing any output, the routing function is called again with error context so +it can select a fallback. + +`RoutedAgent` is different from [workflow agents](workflow-agents/index.md) like +`SequentialAgent` or `ParallelAgent`, which orchestrate multiple agents in a +fixed pattern, and from [LLM-driven +delegation](multi-agents.md#b-llm-driven-delegation-agent-transfer), where the +LLM decides which agent to hand off to. With `RoutedAgent`, you write an +explicit routing function that selects **one** agent per invocation. For +model-level routing, see [Model routing](models/routing.md). + +## How routing works + +Both `RoutedAgent` and [`RoutedLlm`](models/routing.md) are powered by a shared +routing utility that handles selection and failover. + +The router function receives the map of available agents and the current +context, and returns the key of the agent to run. It can be synchronous or +async: + +=== "TypeScript" + + ```typescript + type AgentRouter = ( + agents: Readonly>, + context: InvocationContext, + errorContext?: { failedKeys: ReadonlySet; lastError: unknown }, + ) => Promise | string | undefined; + ``` + +**The `agents` parameter** accepts either a `Record` with +explicit keys, or an array of agents. If an array is provided, each agent's +`name` property is used as its key. + +**Failover behavior:** + +- The router is first called without `errorContext` to make the initial + selection. +- If the selected agent throws an error **before yielding any events**, the + router is called again with `errorContext` containing `failedKeys` and + `lastError`. +- If the selected agent throws an error **after yielding events**, the error + propagates directly without retry, because partial results have already been + emitted. +- A key that has already been tried cannot be re-selected. If the router returns + a previously failed key, the error propagates. +- If the router returns `undefined`, routing stops and the last error is thrown. + +## Basic usage + +Create multiple agents, define a router function that returns a key, and wrap +them in a `RoutedAgent`. The following example routes between two agents based +on an external configuration value that can change between invocations: + +=== "TypeScript" + + ```typescript + --8<-- "examples/typescript/snippets/agents/routing/basic-usage.ts:full" + ``` + +Change `config.selectedAgent` to `'agent_b'` before the next invocation to +route to a different agent. + +## Fallback on error + +When an agent fails, the router is called again with `errorContext` so it can +select a fallback. Failover only applies if the agent fails before yielding any +events (see [How routing works](#how-routing-works)). The following example +checks `errorContext.failedKeys` to avoid re-selecting the failed agent: + +=== "TypeScript" + + ```typescript + --8<-- "examples/typescript/snippets/agents/routing/fallback.ts:config" + ``` + +## Planning mode + +A router can read any external state to select between agents with different +instructions, models, and tools. This lets you implement a planning mode where +the agent switches behavior dynamically. For example, a basic agent might have +read and write tools, while a planning agent is restricted to read-only access +and uses a more powerful model for analysis. + +The following example shows a different `RoutedAgent` configuration. See [basic +usage](#basic-usage) for the full runner setup. + +=== "TypeScript" + + ```typescript + --8<-- "examples/typescript/snippets/agents/routing/planning-mode.ts:config" + ``` + +Set `planningMode = true` before an invocation to route to the planning agent +with its restricted tool set and different instructions. + +## Auto-routing by complexity + +The router function can call a lightweight classifier model to categorize input +and route to different agents accordingly. Because the router can be async, you +can make LLM calls inside it before selecting an agent. + +The following example shows a different `RoutedAgent` configuration. See [basic +usage](#basic-usage) for the full runner setup. + +=== "TypeScript" + + ```typescript + --8<-- "examples/typescript/snippets/agents/routing/auto-routing.ts:config" + ``` diff --git a/docs/api-reference/cli/_sources/index.rst.txt b/docs/api-reference/cli/_sources/index.rst.txt index f081826ccc..99a05d1f5a 100644 --- a/docs/api-reference/cli/_sources/index.rst.txt +++ b/docs/api-reference/cli/_sources/index.rst.txt @@ -1,7 +1,7 @@ ADK CLI documentation ===================== -This page contains the auto-generated command-line reference for ADK 1.31.0. +This page contains the auto-generated command-line reference for ADK 1.32.0. .. contents:: :local: diff --git a/docs/api-reference/cli/_static/documentation_options.js b/docs/api-reference/cli/_static/documentation_options.js index c374b8cc0f..255a4578a8 100644 --- a/docs/api-reference/cli/_static/documentation_options.js +++ b/docs/api-reference/cli/_static/documentation_options.js @@ -1,5 +1,5 @@ const DOCUMENTATION_OPTIONS = { - VERSION: '1.31.0', + VERSION: '1.32.0', LANGUAGE: 'en', COLLAPSE_INDEX: false, BUILDER: 'html', diff --git a/docs/api-reference/cli/genindex.html b/docs/api-reference/cli/genindex.html index 0fe1b7397c..ee7ce0275d 100644 --- a/docs/api-reference/cli/genindex.html +++ b/docs/api-reference/cli/genindex.html @@ -4,11 +4,11 @@ - Index — ADK CLI 1.31.0 documentation + Index — ADK CLI 1.32.0 documentation - + diff --git a/docs/api-reference/cli/index.html b/docs/api-reference/cli/index.html index 2c3a3e4f3c..022a9304a3 100644 --- a/docs/api-reference/cli/index.html +++ b/docs/api-reference/cli/index.html @@ -5,11 +5,11 @@ - ADK CLI documentation — ADK CLI 1.31.0 documentation + ADK CLI documentation — ADK CLI 1.32.0 documentation - + @@ -43,7 +43,7 @@

ADK CLI documentation

-

This page contains the auto-generated command-line reference for ADK 1.31.0.

+

This page contains the auto-generated command-line reference for ADK 1.32.0.