fix: SDXL DoRA LoRA fails with enable_partial_loading=true

[Pfannkuchensack]

Summary

cast_to_device returns a plain torch.Tensor instead of torch.nn.Parameter when moving parameters between devices. This causes _aggregate_patch_parameters to replace valid weights with meta device dummies, falsely triggering DoRA's quantization guard.

Added a check for plain torch.Tensor in _aggregate_patch_parameters so that parameters cast to a different device are recognized as valid non-quantized weights.

Fixes #8624

Related Issues / Discussions

• #8624 — SDXL LoRA (DoRA) fails when enable_partial_loading=true

QA Instructions

1. Load an SDXL model with enable_partial_loading=true
2. Apply a DoRA-based LoRA (e.g. Stabilizer ILNAI)
3. Run a denoise — should complete without the "base model quantization format (likely bitsandbytes) is not compatible with DoRA patches" error

Merge Plan

Single-file change, safe to merge anytime.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[Pfannkuchensack]

https://discord.com/channels/1020123559063990373/1149510134058471514/1494762504599900300

[JPPhoto]

• invokeai/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/custom_module_mixin.py: there is still no targeted regression coverage for the DoRA plus partial-loading plus CPU-to-device autocast path that motivated this PR. A useful follow-up test would exercise a wrapped patched module with base weights on CPU, input and patch on CUDA or MPS, and assert that _aggregate_patch_parameters() preserves real tensors instead of substituting meta.

Otherwise it's good - ping me if you write this test or decide it's too difficult, @Pfannkuchensack.