[API Compatibility] enhance paddle.enable_compat -part - #79391
Conversation
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
发现了需要修复后再合入的兼容性回归，具体问题已放在行级评论中：主要是 paddle.compat 的 lazy attribute 行为被破坏，以及已有的 compat public API 被删除。CI 当前仍有部分任务进行中，且已有 Windows-GPU build/test 失败，请后续一并确认。
| def __getattr__(name): | ||
| if name == "paddle_triton": | ||
| return paddle_triton_fun() |
There was a problem hiding this comment.
这里删掉了 distributions 的 lazy import 和最终的 AttributeError，导致两个回归：paddle.compat.distributions 在未预先 import paddle.compat.distributions 时会通过 __getattr__ 直接返回 None，未知属性也不再抛 AttributeError，会破坏 Python 模块属性协议以及已有的 paddle.compat.distributions 访问方式。请保留 distributions 分支，并让未知属性继续抛 AttributeError。
| def __getattr__(name): | |
| if name == "paddle_triton": | |
| return paddle_triton_fun() | |
| def __getattr__(name): | |
| if name == "paddle_triton": | |
| return paddle_triton_fun() | |
| if name == "distributions": | |
| import importlib | |
| module = importlib.import_module("paddle.compat.distributions") | |
| globals()[name] = module | |
| return module | |
| raise AttributeError(f"module {__name__!r} has no attribute {name!r}") |
| 'BatchNorm1d', | ||
| 'BatchNorm2d', | ||
| 'BatchNorm3d', | ||
| 'MultiheadAttention', |
There was a problem hiding this comment.
这里把 paddle.compat.nn.BatchNorm1D/2D/3D、小写别名以及 SmoothL1Loss 从 public compat surface 删除了，但这些 API 已经有现有调用和测试覆盖：test/legacy_test/test_api_compatibility_part1.py 在类定义时直接绑定 paddle.compat.nn.BatchNorm*，test/legacy_test/test_compat_smooth_l1_loss.py 也直接实例化 paddle.compat.nn.SmoothL1Loss。删除后这些测试会在 import/执行阶段变成 AttributeError，也会破坏已有用户代码。请恢复这些符号及其实现；如果目标是只做命名空间 alias，仍需要保留 compat 侧的实现作为 paddle.* alias 的目标。
建议形状：
__all__ = [
# ...
'BatchNorm1D', 'BatchNorm2D', 'BatchNorm3D',
'BatchNorm1d', 'BatchNorm2d', 'BatchNorm3d',
'SmoothL1Loss',
]
class BatchNorm1D(nn.BatchNorm1D):
... # 保留 torch-style eps/momentum/track_running_stats 适配
BatchNorm1d = BatchNorm1D
# 同步恢复 2D/3D 以及 SmoothL1Loss| return x | ||
|
|
||
| return paddle.nn.functional.unfold( | ||
| return _native(paddle.nn.functional, "unfold")( |
There was a problem hiding this comment.
这里删除 paddle.compat.nn.functional.smooth_l1_loss 会直接破坏已有 compat functional API：test/legacy_test/test_compat_smooth_l1_loss.py 覆盖了 F_compat.smooth_l1_loss(..., beta=...)、target=、size_average/reduce、beta=0 和 forbidden keywords。删除后这些用例和用户迁移代码都会变成 AttributeError，并且 paddle.enable_compat() 也无法把 paddle.nn.functional.smooth_l1_loss alias 到 torch-style 语义。请恢复该函数、__all__ 条目，以及所需的 warnings 和 _ReduceMode。
建议形状：
import warnings
if TYPE_CHECKING:
_ReduceMode: TypeAlias = Literal["mean", "sum", "none"]
__all__ = [
# ...
'unfold',
'smooth_l1_loss',
]
@ForbidKeywordsDecorator(
illegal_keys={"label", "delta", "is_huber", "name"},
func_name="paddle.compat.nn.functional.smooth_l1_loss",
correct_name="paddle.nn.functional.smooth_l1_loss",
)
def smooth_l1_loss(input, target, size_average=None, reduce=None, reduction='mean', beta=1.0):
# 保留原来的 torch-style beta/size_average/reduce 适配逻辑
...
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
CI 分析结果本轮达到时间上限，未完成全部失败原因分析；未分析完的 job 标记为“分析省略/待后续深挖”。 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
liuhao2638
left a comment
liuhao2638
left a comment
There was a problem hiding this comment.
已复查当前提交。此前指出的 paddle.compat lazy attribute、compat.nn BatchNorm/SmoothL1Loss、compat.nn.functional.smooth_l1_loss 删除问题在当前代码中已经恢复。
不过这轮发现了一个新的全局 alias 下的 P1 回归，具体见行级评论：smooth_l1_loss wrapper 需要通过 native 实现回调，避免 enable_compat() 后自引用到 compat wrapper。CI 目前仍有任务运行中，后续也请一并确认。
| for attr_name in getattr(compat_module, PUBLIC_ATTR_DECLARATION): | ||
| if attr_name.startswith("_"): | ||
| continue | ||
| compat_attr = getattr(compat_module, attr_name) |
There was a problem hiding this comment.
这里会把每个 paddle.compat.*.__all__ 符号 alias 到真实 paddle.* namespace，因此全局 paddle.enable_compat() 后 paddle.nn.functional.smooth_l1_loss 也会变成 compat wrapper。当前 wrapper 在函数尾部仍直接调用 paddle.nn.functional.smooth_l1_loss(..., delta=beta, is_huber=False)；alias 生效后这个名字已经指回 wrapper 自身，ForbidKeywordsDecorator 会拒绝 delta/is_huber（或进入自调用路径），导致迁移代码里的 paddle.nn.functional.smooth_l1_loss(x, y, beta=...) 失败。
请像 unfold/SDPA 一样通过 _native 调回 Paddle 原生实现，并补一个 enable_compat() 下的回归用例覆盖这个别名路径。建议形状：
if beta == 0:
return _native(paddle.nn.functional, "l1_loss")(
input, target, reduction=reduction
)
return _native(paddle.nn.functional, "smooth_l1_loss")(
input, target, reduction=reduction, delta=beta, is_huber=False
)
zhwesky2010
changed the title
|
/re-run all-failed |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #79391 +/- ##
==========================================
Coverage ? 95.65%
==========================================
Files ? 6
Lines ? 138
Branches ? 0
==========================================
Hits ? 132
Misses ? 6
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
|
||
|
|
||
| @cache | ||
| def _register_compat_override(): |
There was a problem hiding this comment.
如果 paddle.compat.* 已经patch到 paddle.* 上了，这里还需要去额外操作吗
| # disable_compat() on exit would remove the finder and destroy the | ||
| # surrounding scoped enable (the runtime entry point of the migration | ||
| # workflow). So restore the finder to exactly how it was found. | ||
| if already_has_torch_proxy: |
| public_attrs = getattr(module, PUBLIC_ATTR_DECLARATION) | ||
| torch_module_name = module_info.name.replace( | ||
| PADDLE_PREFIX, TORCH_PREFIX, 1 | ||
| # Use the root-inclusive iterator: pkgutil.walk_packages skips the |
There was a problem hiding this comment.
这些注释清理一下吧，如果确需要注释的话，也只是简短意赅的关键1~2行，不用长篇大论的写
| _restore_paddle_namespace_aliases() | ||
|
|
||
|
|
||
| def enable_compat( |
There was a problem hiding this comment.
直接改这个可能有点不兼容，加个level参数吧
背景这个主要是解决当前的paddle框架中无法完全对齐的问题，当前Paddle API的对齐状态分为以下三种：
用户在编写paddle时，很难知道哪些是对齐的1，哪些是未对齐的2/3，这使得编写paddle代码成本仍较高。 开发思路目前考虑在enable_compat最后面加一个level参数，默认为1，对于：
针对 第三方库+存量Paddle代码 的用法：level=1，部分API无法对齐，需手动检查调整，主要为避免存量paddle代码出现不兼容 针对 第三方库+新增Paddle代码 的用法：level=2，完全对齐，统一按torch的形态即可 @SigureMo 看下这样的设计思路怎么样，有没有要调整的地方？比如 不加level直接按2来、加level但默认为2、新增一个其他API来作为这个开关不用放到enable_compat里？ |
SigureMo
left a comment
SigureMo
left a comment
There was a problem hiding this comment.
针对 第三方库+存量Paddle代码 的用法：level=1，部分API无法对齐，需手动检查调整，主要为避免存量paddle代码出现不兼容
针对 第三方库+新增Paddle代码 的用法：level=2，完全对齐，统一按torch的形态即可
我比较赞同这里的 level=1/2 的方案，主要是考虑到现存代码可能有部分已经使用 Paddle 原生 API 改写过了，此时一旦自动直接将这些 API 迁移到 paddle.compat API 实现可能会出现一些因为兼容性导致的非预期的问题，直接按 2 来我倒是有些担忧的，不过倒是后面可以考虑把 2 切默认（理想情况）
不过值得注意的是，paddle.enable_compat 这个 API 真正的挑战并不在于实现这些映射功能，而在于当环境中真的安装有 PyTorch 时，如何避免发生冲突（CI 中几乎测不到，只有几个使用 fake torch 模拟的 case），这是真实业务场景中会使用的，因此在实现时候需要特别注意下
| return x | ||
|
|
||
| return paddle.nn.functional.unfold( | ||
| return _native(paddle.nn.functional, "unfold")( |
There was a problem hiding this comment.
我有个疑问，既然是 patch，那么一旦开启 compat，所有相关 API 都会变吧？为什么当前改动仅限于 paddle.compat 下？非 compat 下的 API 理论上也会受到影响吧？（只是问题并不是像递归这么明显罢了）
当然不是让你把所有 API 改一遍，而是我在想这样做的可行性
There was a problem hiding this comment.
其实这也很好做，改造一下：
from typing import Callable, TypeVar
from typing_extensions import ParamSpec
from contextlib import contextmanager
T1 = TypeVar("T1")
T2 = TypeVar("T2")
P1 = ParamSpec("P1")
P2 = ParamSpec("P2")
COMPAT_API_ENABLED = False
def is_compat_api_enabled():
return COMPAT_API_ENABLED
@contextmanager
def compat_api_guard(enable: bool = True):
global COMPAT_API_ENABLED
if enable == COMPAT_API_ENABLED:
yield
return
original_value = COMPAT_API_ENABLED
COMPAT_API_ENABLED = enable
try:
yield
finally:
COMPAT_API_ENABLED = original_value
# compat/registry.py
def dispatch_compat_api(
compat_fn: Callable[P1, T1],
) -> Callable[[Callable[P2, T2]], Callable[P2, T2]]:
def native_fn_decorator(native_fn: Callable[P2, T2]) -> Callable[P2, T2]:
def maybe_use_compat_api(*args: P2.args, **kwargs: P2.kwargs) -> T2:
if is_compat_api_enabled():
# 关键在这里，灵活关掉 compat
return compat_api_guard(enable=False)(compat_fn)(*args, **kwargs) # type: ignore
return native_fn(*args, **kwargs)
return maybe_use_compat_api
return native_fn_decorator
def unfold_compat(x, y):
return unfold(x, y) + 1
@dispatch_compat_api(unfold_compat)
def unfold(x, y):
return x * y
with compat_api_guard():
print(unfold(1, 2)) # 3
with compat_api_guard(enable=False):
print(unfold(1, 2)) # 2随便搞的原型，仅供参考
| paddle.enable_compat() | ||
| try: | ||
| self.assertEqual(paddle.min(t).item(), 1.0) | ||
| self.assertEqual(paddle.max(t).item(), 3.0) | ||
| # dim form returns a namedtuple (values, indices) | ||
| r = paddle.min(t, dim=0) | ||
| self.assertEqual(r.values.item(), 1.0) | ||
| self.assertEqual(r.indices.item(), 1) | ||
| finally: | ||
| paddle.disable_compat() |
There was a problem hiding this comment.
这种单测不能直接用 @use_compat_guard 装饰器么？
There was a problem hiding this comment.
@Manfredss 对于整个框架内部，调用到这些compat系列API的地方（约30个），确实应该都是paddle naive版的，而这一系列API对外呈现的则是torch版的。
这个主要影响到的是 单测 + 部分调用paddle API的组合实现API。
可能得加大一下测试面，比如level=2下，paddle其他的API还能否跑过，除了paddle自身测试外，建议在paconvert里也进行测试（全局设置为level=2），测量全量API的行为是否正常。
|
paddle.enable_compat 增加 level 参数： 背景：Paddle API 对齐分三类:
设计：
level=2 关键实现
测试
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
/re-run all-failed |
There was a problem hiding this comment.
Pull request overview
This PR extends paddle.enable_compat() with a new compatibility “level=2” mode that (when enabled globally) aliases public paddle.compat.* APIs onto the live paddle / paddle.nn / paddle.nn.functional namespaces, using caller-aware dispatch to keep Paddle internal calls on native implementations. It also adds guards to avoid recursion when compat wrappers call back into aliased paddle.* APIs, and introduces comprehensive tests for aliasing + lifecycle behavior.
Changes:
- Add a level-based compat mode where global
enable_compat(level=2)aliasespaddle.*topaddle.compat.*using caller-aware dispatchers/proxies (and restores exactly on disable). - Add
compat_api_guard(enable=False)usage in compat wrappers to ensure internal calls hit native implementations under aliasing. - Add new test coverage (including fake modules) for namespace aliasing, scoped/global lifecycle behavior, and torch root API mapping.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/compat/test_compat_namespace_aliased.py | New end-to-end tests for level=2 aliasing, restoration, lifecycle correctness, and caller-aware dispatch behavior. |
| test/compat/fake_modules/torch_proxy_root_api_module.py | Fake module used to validate scoped imports + torch root API capture behavior. |
| python/paddle/compat/proxy.py | Implements level=2 namespace aliasing, caller-aware dispatch, compat suspension guard, root-package override coverage, and improved guard restoration logic. |
| python/paddle/compat/nn/transformer.py | Applies compat_api_guard(enable=False) to internal paths that must remain native under aliasing. |
| python/paddle/compat/nn/functional/sdpa.py | Guards compat SDPA implementation so its internals consistently use native implementations. |
| python/paddle/compat/nn/functional/init.py | Guards compat unfold to ensure internal calls remain native under level=2 aliasing. |
| python/paddle/compat/nn/init.py | Guards compat AvgPool* forward; adjusts Softmax.extra_repr (but currently introduces a bug). |
| python/paddle/compat/init.py | Exposes compat dispatch/guard helpers and guards several top-level compat functions to prevent recursion under aliasing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def extra_repr(self) -> str: | ||
| return f"dim={self._dim}" | ||
| return f"dim={self.dim}" | ||
|
|
| # (live module, attr name) -> original value (or ``_MISSING``) for every | ||
| # ``paddle.*`` symbol aliased to its ``paddle.compat.*`` counterpart, so the | ||
| # paddle namespace can be restored exactly on ``disable_compat()``. | ||
| _PADDLE_NAMESPACE_SAVED: dict[tuple[types.ModuleType, str], Any] = {} | ||
|
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
/re-run all-failed |
|
/re-run all-failed |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
/re-run all-failed |
| return super().__new__(mcls, name, bases, namespace, **kwargs) | ||
|
|
||
| def __instancecheck__(cls, instance: object) -> bool: | ||
| native_cls = cls.__dict__.get('__native_cls__') |
There was a problem hiding this comment.
针对普通场景没有使用enable_compat(2)下的 isinstance，没有覆盖测试
There was a problem hiding this comment.
已补上：test_class_type_compatibility_without_alias 现在直接在未启用 enable_compat(2) 的情况下构造 compat_cls / native_cls，并断言 native 不是 compat_cls 的实例，所以普通场景的 isinstance / issubclass 也已经有覆盖。
| yield compat_module | ||
|
|
||
|
|
||
| def _make_caller_aware_class_proxy(native_cls: type, compat_cls: type) -> type: |
There was a problem hiding this comment.
你这个名字可能得改下，上面那个叫 dispatch_function，这个叫 dispatch_class比较清晰
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查当前提交，之前的 class type-checking concern 和普通场景的 isinstance 覆盖都已补齐，未发现新的阻塞问题。
|
/re-run all-failed |
|
/re-run all-failed |
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查新提交。此前 class type-checking 相关 concern 仍保持已处理状态；这次 root paddle.compat 注册改动让默认 level=1 的 torch.* root API 语义发生回退，详细说明见行级评论，需要修复后再推进。
我做了 git diff --check 和相关 Python 文件的语法编译，均通过；完整 Paddle 单测在当前源码目录缺少已安装 wheel / libpaddle，未能本地执行。
| for module in _iter_compat_modules(): | ||
| torch_module_name = module.__name__.replace("paddle.compat", "torch", 1) | ||
| for attr_name in module.__all__: | ||
| if attr_name.startswith("_"): | ||
| continue | ||
| paddle_attr = getattr(module, attr_name) | ||
| torch_attr_name = f"{torch_module_name}.{attr_name}" | ||
| compat_overrides[torch_attr_name] = RawOverriddenAttribute( | ||
| paddle_attr |
There was a problem hiding this comment.
处理要求：请针对该评论修复并提交新的 commit。
这里复用 _iter_compat_modules() 后，root paddle.compat.__all__ 也会被无条件注册进 GLOBAL_OVERRIDES。enable_compat() 仍默认 level=1，并且每次都会先调用 _register_compat_override()，所以默认 level=1 下 import torch; torch.sort/min/unique 会直接变成 paddle.compat.*，torch.slogdet 也会可见。这个结果改变了既有默认 proxy 语义，也和前面已确认的“level=1 不把 root __all__ 塞进全局覆盖表”相反。
请保留子模块 compat override 的无条件注册，但跳过 root paddle.compat；root compat-only API（例如 slogdet）只在 level=2 的 root proxy 构造时临时补到 overrides。建议形状：
def _register_compat_override():
compat_overrides = {}
for module in _iter_compat_modules():
if module.__name__ == "paddle.compat":
continue
...并在 _find_spec_for_torch_module() 中只针对 fullname == "torch" and len(_PADDLE_NAMESPACE_SAVED) > 0 补充 paddle.compat.__all__ 里没有 native paddle.* 对应项的符号。
There was a problem hiding this comment.
也和前面已确认的“
level=1不把 root__all__塞进全局覆盖表”相反。
谁确认的？不是说了 level=1 要加上吗？加了有什么问题吗？
There was a problem hiding this comment.
这里是我理解偏差了。我引用的是前面 Manfredss 在这条链路里的回复，以及我随后按“level=1 保持 root torch.* -> paddle.* 原映射、只在 level=2 补 root compat-only”收窄后的结论；但如果维护者期望是 level=1 下 root torch.sort/min/unique/slogdet 也直接走 paddle.compat.*，那当前这版把 root paddle.compat.__all__ 注册进 torch proxy 是符合该语义的。
所以我撤回这条 P1，不再把它作为阻塞项。后续只需要确保 PR 描述和测试语义保持一致即可。
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已按维护者反馈复查，上一条关于 level=1 root torch.* override 的 P1 是我对目标语义理解偏差，已在对应 thread 中撤回，不再作为阻塞项。当前未发现需要阻塞合入的问题。
|
/re-run all-failed |
1 similar comment
|
/re-run all-failed |
SigureMo
changed the title
paddle.enable_compat -part
|
@ShigureNyako 来看看 Static-Check 为什么明明 approve 了还是失败 |
risemeup1111
left a comment
risemeup1111
left a comment
There was a problem hiding this comment.
已复查，当前代码 head 未变化，未发现新的阻塞问题。仅有一处 PR 描述需要和最新确认的语义同步。
-
优先级：P3 非行级：PR 描述不在 changed diff line 上。当前描述仍写“
slogdet只通过 Torch proxy 在level=2暴露”以及“existinglevel=1users unaffected”，但最新代码和维护者确认的语义是 roottorch.sort/min/unique/slogdet在level=1也直接走paddle.compat.*，level=2额外开启paddle.*/paddle.Tensornamespace dispatch。建议把描述改成这层区别，避免后续按旧语义误读。处理要求：请针对该评论进行回复（同意并已修改请回复 Done，不同意请说明理由）。
|
@risemeup1111 针对 这条 P3 review，当前语义确认如下： |
|
@SigureMo 关于 Static-Check 失败的提问： |
|
@ShigureNyako 收到。按这个语义确认即可： 这条回复对前面的 P3 来说已经足够，我把该项按已回复处理；本轮不再要求修改 PR 描述，也不作为后续阻塞项。 |
PR Category
User Experience
PR Types
Improvements
Description
This PR adds an optional compatibility level to
paddle.enable_compat.level=1remains the default and preserves the existing compatibility behavior.level=2includes alllevel=1behavior and additionally dispatches alignedpaddle.compat.*APIs through their correspondingpaddle.*namespaces. Only public compat APIs declared in__all__participate in this dispatch. Calls from user code use the compat implementation, while Paddle-internal callscontinue to use the original native implementation. Class APIs retain their class, inheritance, and type-checking semantics. Compat APIs without a corresponding
paddle.*API, currentlyslogdet, are exposed only through the Torch proxy inlevel=2.paddle.disable_compat()restores all affected Paddle namespace entries. Therefore, existinglevel=1users and previously passing tests are unaffected. Targeted tests cover namespace dispatch, restoration, class APIs, andslogdet.是否引起精度变化
否