feat(apionly): seedream provider#9054
Open
Pfannkuchensack wants to merge 66 commits intoinvoke-ai:mainfrom
Open
feat(apionly): seedream provider#9054Pfannkuchensack wants to merge 66 commits intoinvoke-ai:mainfrom
Pfannkuchensack wants to merge 66 commits intoinvoke-ai:mainfrom
Conversation
Add BytePlus Seedream as external image generation provider with four models (seedream-4.5, seedream-4.0, seedream-3.0-t2i) including debug dump support and batch generation. Hide irrelevant canvas settings for external models by using ExternalModelCapabilities as the source of truth for UI rendering. Scheduler, LoRA, CFG Scale, and all Advanced settings (VAE, CLIP Skip, Seamless, etc.) are now hidden for external models. Steps, Guidance, and Seed controls are only shown when the model declares support via its capability flags. Adds supports_steps capability field and gates the graph builder accordingly.
…pi_keys.yaml Add 'external', 'external_image_generator', and 'external_api' to Zod enum schemas (zBaseModelType, zModelType, zModelFormat) to match the generated OpenAPI types. Remove redundant union workarounds from component prop types and Record definitions. Fix type errors in ModelEdit (react-hook-form Control invariance), parsing.tsx (model identifier narrowing), buildExternalGraph (edge typing), and ModelSettings import/export buttons. Move external_gemini_base_url and external_openai_base_url into api_keys.yaml alongside the API keys so all external provider config lives in one dedicated file, separate from invokeai.yaml.
Add combined resolution preset selector for external models that maps aspect ratio + image size to fixed dimensions. Gemini 3 Pro and 3.1 Flash now send imageConfig (aspectRatio + imageSize) via generationConfig instead of text-based aspect ratio hints used by Gemini 2.5 Flash. Backend: ExternalResolutionPreset model, resolution_presets capability field, image_size on ExternalGenerationRequest, and Gemini provider imageConfig logic. Frontend: ExternalSettingsAccordion with combo resolution select, dimension slider disabling for fixed-size models, and panel schema constraint wiring for Steps/Guidance/Seed controls.
- Remove negative_prompt, steps, guidance, reference_image_weights, reference_image_modes from external model nodes (unused by any provider) - Remove supports_negative_prompt, supports_steps, supports_guidance from ExternalModelCapabilities - Add provider_options dict to ExternalGenerationRequest for provider-specific parameters - Add OpenAI-specific fields: quality, background, input_fidelity - Add Gemini-specific fields: temperature, thinking_level - Add new OpenAI starter models: GPT Image 1.5, GPT Image 1 Mini, DALL-E 3, DALL-E 2 - Fix OpenAI provider to use output_format (GPT Image) vs response_format (DALL-E) and send model ID in requests - Add fixed aspect ratio sizes for OpenAI models (bucketing) - Add ExternalProviderRateLimitError with retry logic for 429 responses - Add provider-specific UI components in ExternalSettingsAccordion - Simplify ParamSteps/ParamGuidance by removing dead external overrides - Update all backend and frontend tests
github-actions
Bot
added
api
python
- Gemini recall: write temperature, thinking_level, image_size to image metadata; wire external graph as metadata receiver; add recall handlers. - Canvas: gate regional guidance, inpaint mask, and control layer for external models. - Canvas: throw a clear error on outpainting for external models (was falling back to inpaint and hitting an API-side mask/image size mismatch). - Workflow editor: add ui_model_provider_id filter so OpenAI and Gemini nodes only list their own provider's models. - Workflow editor: silently drop seed when the selected model does not support it instead of raising a capability error. - Remove the legacy external_image_generation invocation and the graph-builder fallback; providers must register a dedicated node. - Regenerate schema.ts. - remove Gemini debug dumps to outputs/external_debug
# Conflicts: # invokeai/app/invocations/external_image_generation.py # invokeai/frontend/web/src/services/api/schema.ts
- Export imageSizeChanged from paramsSlice so the metadata recall handler can import it. - Build the external graph's metadata model entry via zModelIdentifierField (ExternalApiModelConfig is not in the AnyModelConfig union). - Strip Seedream debug payload/image dumps. - Regenerate schema.ts.
Collaborator
|
Functional testing only so far. Here's what I noticed. Seedream 3.0 T2I modelI can't get this one to run at all. All attempts give me Seedream 4.0 model
Seedream 4.5 modeltext2img is working. However, I have the same issues with reference images, init image and inpaint mask as with the 4.0 model. Seedream 5.0 liteI can't get this model to work. I'm getting the "does not exist or you do not have access to it" error message. Location of Seedream API keyI notice that the Seedream API key is being stored in |
Collaborator
|
@Pfannkuchensack I'm afraid there are now a lot of conflicts resulting from (I think) the earlier merge of #8884 . |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds BytePlus Seedream as an external image generation provider on top of the external-models base (PR #8884). Introduces four Seedream starter models (5.0 Lite, 4.5, 4.0, 3.0 T2I) with capability-driven settings visibility, reference-image support, and provider-specific UI options (watermark toggle, prompt optimization).
What:
invokeai/app/services/external_generation/providers/seedream.pySeedreamProviderOptions.tsx— watermark & prompt-optimization togglesWhy:
Seedream offers high-quality multi-reference generation at 2K/4K, which fills a gap in the current external-provider lineup.
How:
Follows the external-provider pattern established in PR #8884. Capabilities on each
StarterModeldrive which settings panels render, so no UI-side branching is needed beyond the Seedream-only options.Related Issues / Discussions
QA Instructions
api_keys.yaml.SEEDREAM_ASPECT_RATIOS.Merge Plan
Rebase/merge after #8884 lands. No DB migrations. Config-default changes are additive.
Checklist
What's Newcopy (if doing a release after this PR)