Add HostedToolSearchTool and SearchableAIFunctionDeclaration for tool search / deferred loading support

[Copilot]

Implements tool search and deferred loading support (issue #7371): a HostedToolSearchTool marker and SearchableAIFunctionDeclaration decorator that signals per-tool deferred loading to supporting providers. OpenAI Responses API support included; Anthropic follows separately.

New abstractions (Microsoft.Extensions.AI.Abstractions)

• HostedToolSearchTool — marker AITool (same pattern as HostedWebSearchTool/HostedCodeInterpreterTool); maps to the tool_search hosted tool.
• SearchableAIFunctionDeclaration — decorator wrapping AIFunctionDeclaration that signals deferred loading to supporting providers. Includes optional Namespace property for grouping related tools and a CreateToolSet convenience method for creating a complete tool list.
• DelegatingAIFunctionDeclaration — now public (was internal); base class for SearchableAIFunctionDeclaration that passes through calls to an inner AIFunctionDeclaration.
• HostedMcpServerTool.DeferLoadingTools — new [Experimental] bool property that signals to the provider that tools from this MCP server should use deferred loading when tool search is enabled.
• DiagnosticIds.Experiments.AIToolSearch — new constant (maps to existing MEAI001)

All new types are [Experimental(AIToolSearch)].

OpenAI provider (Microsoft.Extensions.AI.OpenAI)

• HostedToolSearchTool → deserialized ResponseTool from {"type":"tool_search"} via AOT-safe ModelReaderWriter.Read with OpenAIContext.Default (created per-call, not cached; temporary workaround for openai/openai-dotnet#1053)
• Deferred loading for functions: provider detects SearchableAIFunctionDeclaration via tool.GetService<SearchableAIFunctionDeclaration>() and patches defer_loading: true and optional namespace onto matching FunctionTool instances
• Deferred loading for MCP servers: provider patches defer_loading: true onto McpTool when HostedMcpServerTool.DeferLoadingTools is true
• Plain (non-searchable) AIFunctionDeclaration tools are sent without deferred loading

Tests

• Unit tests (OpenAIResponseClientTests.cs): VerbatimHttpHandler-based tests validating the exact JSON request body sent for various configurations: tool_search only, searchable functions with deferred loading, mixed searchable and plain functions, and mixed with other hosted tools (web search)
• Conversion tests (OpenAIConversionTests.cs): Tests for AsOpenAIResponseTool extension covering tool_search conversion, defer_loading patching via SearchableAIFunctionDeclaration, namespace patching, and plain function without deferred loading
• Abstractions tests (HostedToolSearchToolTests.cs, SearchableAIFunctionDeclarationTests.cs): Tests for tool properties, delegation, GetService discovery, CreateToolSet, and namespace roundtripping
• Integration test (OpenAIResponseClientIntegrationTests.cs): UseToolSearch_WithDeferredFunctions test exercising HostedToolSearchTool with SearchableAIFunctionDeclaration-wrapped functions against the OpenAI API

Usage

// Wrap specific functions as searchable (deferred loading)
var options = new ChatOptions
{
    Tools =
    [
        new HostedToolSearchTool(),
        new SearchableAIFunctionDeclaration(getWeather),
        new SearchableAIFunctionDeclaration(getForecast),
        importantTool, // not wrapped — sent without defer_loading
    ]
};

// Use CreateToolSet helper to wrap all functions at once
var tools = SearchableAIFunctionDeclaration.CreateToolSet(
    [getWeather, getForecast, getNews],
    @namespace: "weather");

var response = await chatClient.GetResponseAsync(messages, new() { Tools = tools });

// Enable deferred loading for an MCP server's tools
var mcpTool = new HostedMcpServerTool("myServer", "https://example.com/mcp")
{
    DeferLoadingTools = true
};

Original prompt

Problem

Implement tool search and deferred loading support as described in #7371. Both OpenAI and Anthropic now support tool search, where tool definitions can be sent with deferred loading (only name/description sent upfront, full schema deferred) and a special tool_search hosted tool is included that the model can invoke to search for and load full tool definitions on demand.

Design

Follow Option A from the issue discussion — a HostedToolSearchTool marker tool + a SearchableAIFunctionDeclaration decorator, consistent with existing patterns (HostedWebSearchTool, ApprovalRequiredAIFunction, etc.).

Requirements

1. New types in Microsoft.Extensions.AI.Abstractions

HostedToolSearchTool (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs)

• A new AITool subclass, following the exact same pattern as HostedWebSearchTool and HostedCodeInterpreterTool.
• Name returns "tool_search".
• Has the same constructor pattern (default + one taking IReadOnlyDictionary<string, object?>? additionalProperties).
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].

SearchableAIFunctionDeclaration (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Functions/SearchableAIFunctionDeclaration.cs)

• A new class that extends DelegatingAIFunctionDeclaration (which is currently internal). Important: DelegatingAIFunctionDeclaration is the declaration-only delegating base (not DelegatingAIFunction which requires AIFunction). This is because SearchableAIFunctionDeclaration should work with AIFunctionDeclaration instances that may not have InvokeAsync.
• Constructor takes AIFunctionDeclaration innerFunction and string? namespaceName = null.
• Has a Namespace property (string?) for grouping related tools.
• Must be sealed.
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].
• Include a static convenience helper method: public static IList<AITool> CreateToolSet(IEnumerable<AIFunctionDeclaration> functions, string? namespaceName = null, IReadOnlyDictionary<string, object?>? toolSearchProperties = null) that wraps all functions as SearchableAIFunctionDeclaration and prepends a HostedToolSearchTool, returning a complete tool list ready for ChatOptions.Tools.

2. DiagnosticIds update

In src/Shared/DiagnosticIds/DiagnosticIds.cs, add a new constant in the Experiments class:

internal const string AIToolSearch = AIExperiments;

Place it alongside the other AI experiment constants (near AIWebSearch, AICodeInterpreter, etc.).

3. OpenAI provider implementation

In src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs:

In ToResponseTool(AITool tool, ChatOptions? options = null) method:

Add handling for HostedToolSearchTool — this maps to the OpenAI tool_search response tool. Since the underlying OpenAI .NET SDK likely doesn't have a ToolSearchTool class yet, you need to manually construct a ResponseTool from JSON. Cache the deserialized ResponseTool instance in a static field so it's only created once. Use the ModelReaderWriter pattern or direct JSON deserialization to create a ResponseTool from the JSON {"type": "tool_search"}. Pattern:

case HostedToolSearchTool:
    return s_toolSearchResponseTool ??= ModelReaderWriter.Read<ResponseTool>(BinaryData.FromString("""{"type": "tool_search"}"""))!;

Add a private static ResponseTool? s_toolSearchResponseTool; field to cache it.

For SearchableAIFunctionDeclaration: When an AIFunctionDeclaration is detected as having a SearchableAIFunctionDeclaration via GetService<SearchableAIFunctionDeclaration>(), the generated FunctionTool should have defer_loading set to true and optionally include the namespace metadata. Since the OpenAI SDK's FunctionTool class may not have these properties yet, use the Patch property to set them on the JSON. The check should happen in the existing case AIFunctionDeclaration aiFunction: branch — after calling ToResponseTool(aiFunction, options), check if the original tool (or aiFunction) has GetService<SearchableAIFunctionDeclaration>() and if so, patch the resulting FunctionTool with defer_loading and namespace. This is done in the ToResponseTool(AITool, ChatOptions?) method so it doesn't infect the general ToResponseTool(AIFunctionDeclaration, ChatOptions?) helper. Specifically, the case AIFunctionDeclaration aiFunction: case should become:

case AIFunctionDeclaration aiFunction:
    var functionTool = ToResponseTool(aiFunction, options);
    if (tool.GetService<SearchableAIFunctionDeclaration>() is { } searchable)
    {
        functionTool.Patch.Set("$.defer_loading"u8, "true"u8);
        if (searchable.Namespace is { } ns)
        {
   ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/extensions/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

 ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/dotnet/extensions/pull/7377)

[stephentoub]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of either AITool instances or string names, one for Enable and one for Disable. By default, both of these collections would be null, in which case it would treat every tool as Enable and mark every tool as defer-loading. If Disable was non-null but Enable was null, then it would treat every tool as Enable except for those listed in Disable. Conversely if Enable was non-null and Disable was null, it would only treat those tools in Enable as defer-loading, and if both were non-null, it would treat everything in Enable as defer-loading unless it was also listed in Disable. Thoughts? Is the dominant case for tool search with OpenAI and Anthropic one where most tools are in the same boat and only zero to a small number are special-cased the other way? Or is the right way to model this what we currently have?

[Copilot]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of ei...

Yes, I think the dominant case is indeed "most/all tools are searchable" — the whole point of tool search is to scale to many tools, so users will almost always want blanket defer-loading with at most a few exceptions.

Your proposed design is better for that common case. Today, with 50 tools you'd wrap all 50 in SearchableAIFunctionDeclaration; with the proposed design, you'd just add new HostedToolSearchTool() and you're done. Much cleaner.

A few thoughts on the details:

Enable/Disable semantics: The logic makes sense and is consistent with how HostedMcpServerTool.AllowedTools works (null = all allowed). I wonder if the naming could be more specific though — maybe DeferredTools / NonDeferredTools, or SearchableTools / AlwaysLoadedTools? The Enable/Disable naming is generic and doesn't hint at what is being enabled. But that's a bikeshed.

String names vs AITool references: String names (like AllowedTools uses) are simpler and avoid lifetime/reference issues, but AITool references give type safety. Either could work; strings are probably simpler since the provider just needs to match tool names when iterating ChatOptions.Tools.

Namespace: The current design has per-tool Namespace. If we consolidate into HostedToolSearchTool, we'd need either: (a) a single Namespace property on HostedToolSearchTool that applies to all searchable tools, (b) a Dictionary<string, IList<string>> mapping namespaces to tool names, or (c) drop namespace for now and add it later if needed. I'd lean toward (a) or (c) — a single namespace is probably fine for most uses, and (c) keeps the API surface minimal while we learn more.

What we'd lose: The GetService<SearchableAIFunctionDeclaration>() discovery pattern, but that's only useful in middleware, and middleware doesn't typically need to know about defer-loading (it's a wire-level optimization). DelegatingAIFunctionDeclaration would also no longer need to be public (can revert to internal).

Overall I think this is a better design. Want me to implement it?

[stephentoub]

Want me to implement it?

@copilot, yes, please

[Copilot]

Want me to implement it?

@copilot, yes, please

Implemented in f295d10.

Changes:

• HostedToolSearchTool now has DeferredTools and NonDeferredTools (IList<string>?) properties with the enable/disable semantics you described (both null = all deferred)
• Removed SearchableAIFunctionDeclaration entirely
• Reverted DelegatingAIFunctionDeclaration to internal
• OpenAI provider finds HostedToolSearchTool from ChatOptions.Tools and applies defer_loading: true to matching function tools
• Used AOT-safe ModelReaderWriter.Read<ResponseTool>(..., OpenAIContext.Default) and "true"u8 literal (no pragma suppressions)
• Added ChatOptions? parameter to AsOpenAIResponseTool extension (follows AsOpenAIResponseTextFormat pattern)

[stephentoub]

@PederHP any thoughts?

[Copilot]

Pull request overview

Adds a new experimental hosted tool marker (HostedToolSearchTool) to enable “tool_search” and implements deferred loading (defer_loading) support in the OpenAI Responses provider by patching function tools based on DeferredTools / NonDeferredTools configuration.

Changes:

• Introduces HostedToolSearchTool (experimental) and a new DiagnosticIds.Experiments.AIToolSearch constant.
• Updates OpenAI Responses tool conversion to emit tool_search and optionally patch function tools with defer_loading: true.
• Adds/extends unit tests covering tool_search conversion, caching, and deferred-loading selection logic.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIConversionTests.cs	Adds OpenAI conversion tests for tool_search and defer_loading behavior.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Tools/HostedToolSearchToolTests.cs	Adds tests for HostedToolSearchTool construction and list properties.
src/Shared/DiagnosticIds/DiagnosticIds.cs	Adds DiagnosticIds.Experiments.AIToolSearch.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs	Adds tool_search mapping and deferred-loading patching for function tools.
src/Libraries/Microsoft.Extensions.AI.OpenAI/MicrosoftExtensionsAIResponsesExtensions.cs	Updates AsOpenAIResponseTool to accept ChatOptions? for deferred-loading context.
src/Libraries/Microsoft.Extensions.AI.OpenAI/Microsoft.Extensions.AI.OpenAI.json	Updates API baseline for the modified extension method signature.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs	Adds the new experimental HostedToolSearchTool abstraction with configuration lists.

[stephentoub]

@copilot why was it valuable to change the signature of ToResponseTool like this? I'm not seeing the benefits rather than just having ToResponseTool call FindToolSearchTool if/when it needs to.

[stephentoub]

Oh, nevermind, I see, because we need to call ToResponseTool for each AIFunction

[Copilot]

You're right — there was no benefit to exposing the HostedToolSearchTool? parameter on the internal ToResponseTool. Fixed in b53a6b9: added a simpler ToResponseTool(AITool, ChatOptions?) overload that calls FindToolSearchTool internally. The 3-param version and FindToolSearchTool are now both private. The internal loop still pre-finds the tool for O(N) efficiency.

[Copilot]

@copilot there's plenty of unrelated changes, especially to eng files, can you please investigate and fix it.

Addressed in 54d26f8 — reset 220+ unrelated files (eng/, .github/, etc.) back to match main. The original merge commit (14dbaca) brought in all of main's changes into the PR diff. Most unrelated files are now cleaned up, though a few residual version-bump diffs remain that would need a full rebase to eliminate completely.

[jozkee]

would need a full rebase to eliminate completely.

@copilot do that please.

[Copilot]

would need a full rebase to eliminate completely.

@copilot do that please.

Fixed in a4a92c3 — all non-tool-search files have been reset to match main. The PR diff now contains exactly 9 files (641 insertions, 2 deletions), all directly related to the tool search feature.

[jozkee]

@copilot address feedback

[jozkee]

Suggested change

	public SearchableAIFunctionDeclaration(AIFunctionDeclaration innerFunction, string? namespaceName = null)
	public SearchableAIFunctionDeclaration(AIFunctionDeclaration innerFunction, string? namespace = null)

[Copilot]

Done in 74f0c51 — renamed namespaceName to @namespace in both the constructor and CreateToolSet.

[jozkee]

Suggested change

	string? namespaceName = null,
	string? namespace = null,

[Copilot]

Done in 74f0c51.

[jozkee]

Add a comment referencing openai/openai-dotnet#1053 pointing out this is a temporary workaround.

[Copilot]

Done in 74f0c51 — added a comment referencing the issue above the ModelReaderWriter.Read call.

[jozkee]

Add bool DeferLoadingTools annotated as Experimental to HostedMcpServerTool with remarks explaining that this is related to Tool Search.

[Copilot]

Done in 74f0c51 — added [Experimental(AIToolSearch)] bool DeferLoadingTools to HostedMcpServerTool with remarks explaining it signals deferred loading when tool search is enabled. The OpenAI provider applies defer_loading via Patch when this property is true.

[Copilot]

@copilot address feedback

All feedback addressed in 74f0c51.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

[Copilot]

SearchableAIFunctionDeclaration is documented as enabling deferred loading where only name/description are sent initially, but the OpenAI conversion still always includes full function parameters (via ToResponseTool(aiFunction, ...)). This likely defeats the token-savings goal of deferred loading and doesn’t match the new abstractions’ remarks. Consider emitting a minimal FunctionTool payload (omitting/deferring the schema) when SearchableAIFunctionDeclaration is detected, or adjust the documentation/behavior consistently.

[Copilot]

Creating the tool_search ResponseTool allocates a new JSON string/BinaryData each time via BinaryData.FromString(...). Even if you intentionally avoid caching the deserialized ResponseTool, consider caching the serialized payload (e.g., static BinaryData/UTF8 bytes) to reduce per-call allocations in tool conversion.

[jozkee]

This is not how namespaces work in OpenAI, they are a tool array as a tool itself. Anthropic doesn't have such concept.