Feature: Expose the underlying model API response headers

[cedrictwillie]

I couldn't find where the underlying response was accessible. My use case is being able to inspect the response headers for rate limit quota information.

• x-ratelimit-limit-requests
• x-ratelimit-limit-tokens
• x-ratelimit-remaining-requests
• x-ratelimit-remaining-tokens
• etc.

OpenAI: https://developers.openai.com/api/docs/guides/rate-limits#rate-limits-in-headers

[dumbbellcode]

is it something the team is interested in? then i can work on it.

ping: @pushpak1300

[pushpak1300]

How do you think an API looks for this? I’m thinking of providing raw resposne object on the result.

[cedrictwillie]

@pushpak1300 @dumbbellcode The API could be similar to how "steps" is constructed in the response. Where there is a steps array containing all steps, and the top level response structure contains the final step. This covers single/multi turn calls.

Or, there could be an option to hook into or configure the HTTP client that will be used entirely? Just throwing out initial ideas. Honestly haven't looked into the code deeply to understand the implications of these ideas.

use Illuminate\Support\Facades\Http;

class FooAgent implements Agent
{
    protected function withHttpClient()
    {
        return Http::baseUrl('')
            ->withHeaders(['...'])
            ->throw();
    }
}

[dumbbellcode]

@pushpak1300 @cedrictwillie

Option 1: we can expose the last http call response headers in the TextResponse class for example

$response = $agent->prompt('Summarise this document.')
$headers = $response->headers // accessible from the last api call

Option 2: we can expose the whole response object from the last http call, as it's available

$response = $agent->prompt('Summarise this document.')
$httpResponse = $response->httpResponse // accessible from the last api call

personally i'd suggest we expose the whole object (option 2), without any kind of opt-in from the users, as it's readily available after api calls

[pushpak1300]

$response->raw make more sense to me ? Are we always gonna store the latest response ?

[cedrictwillie]

I like $response->raw! Having "hooks" into every response is the main concern I was thinking of when suggesting the ability to configure the HTTP client that will be used. This give the ability to configure middleware.

This definitely complicates the implementation, so maybe just the latest response is suitable for now? ($response->raw)

[dumbbellcode]

agreed, simplest for now is having the latest raw reponse in textresponse object, going ahead with the change.

[wadakatu]

Happy to pick this up — it's been open a while, and it would also cover the native response-id need from #675 (which was closed in favor of this).

@pushpak1300 before I open a PR, could you confirm the shape you'd prefer? Based on the discussion above I'm planning:

• expose $response->raw on TextResponse (and StructuredTextResponse)
• holding the latest raw provider response as the decoded array (response body)

A couple of open points from the thread:

1. Array vs object — decoded array ($response->raw['id'], …), or the full Illuminate\Http\Client\Response so headers are reachable too?
2. Latest only, or per-step — also surface the raw response on each Step for multi-step/tool loops, or just the final one for now?

Once you confirm, I'll open the PR. cc @dumbbellcode in case you're still planning to take it — happy to coordinate or hand off.

[dumbbellcode]

really sorry for this @wadakatu @pushpak1300, I had committed the changes but got pulled into other things and didn't raise the PR, the PR is ready now 🙂

#714