feat(py): box background operation before wrap_model - #6130
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request introduces support for background models (long-running operations) in Genkit. It updates run_one_iteration to handle Operation responses by wrapping them in a ModelResponse before middleware execution, and automatically sets the longRunning capability flag on background models during registration. Additionally, comprehensive tests are added to verify this behavior. The review feedback highlights two robustness improvements: handling cases where the raw response is a dictionary (e.g., over the wire) by deserializing it into an Operation or ModelResponse, and ensuring that if the model's supports field is a Pydantic model rather than a dictionary, it is correctly serialized instead of being overwritten.
05b81df to
d01e7a6
Compare
d01e7a6 to
4a15ddf
Compare
4a15ddf to
260240f
Compare
5ac1bc9 to
1eae4b2
Compare
51d5429 to
2af5484
Compare
2af5484 to
26276a6
Compare
Summary
generate()on a background model returns aModelResponsewhoseoperationis the start handle and whosemessageisNone. It does not poll until done.generate_operation()is that same start and returns theOperation.await next_fn(...)inwrap_modelis always aModelResponse, so a hook that reads.messageis safe.ActionKind.BACKGROUND_MODEL, notoperation is not None. Adefine_modelLRO that also returns a tool request still runs the tool loop.wrap_modelthat rebuildsModelResponsewithout forwardingoperationisFAILED_PRECONDITION(pass through response.operation). It does not blame the model.wrap_generaterebuild that dropsoperationis the same status with awrap_generatemessage. We do not split “middleware ate it” from “start never had one.”ModelResponse.__eq__includesoperation, so two start handles with different job ids are not equal.