Split `GroundingDINO` Fixes, Use Package

[phreed]

The goal is to:

• merge the relevant updates from SelectiveTracking into GroundingDINO,
• remove GroundingDINO from SelectiveTracking,
• build a set of conda packages from the updated GroundingDINO via the conda-forge feedstock,
• use that conda package in the updated SelectiveTracking.

Relevant git repositories:

conda-forge/groundingdino-py-feedstock#2 : the conda-forge feedstock
https://pypi.org/project/groundeddino-vl/
https://pypi.org/project/rf-groundingdino/
https://pypi.org/project/groundingdino-mod/
https://pypi.org/project/groundino-samnet/
https://pypi.org/project/groundingdino-gml/
https://pypi.org/project/groundingdino-iscas/
https://pypi.org/project/groundingdino-yl/
https://pypi.org/project/groundingdino-stk/
https://pypi.org/project/groundingdino-py/ : I believe this is what the conda-forge package is based upon
https://pypi.org/project/nw-groundingdino/

[phreed]

There are numerous forks of GroundingDINO
https://github.com/IDEA-Research/GroundingDINO/forks

[phreed]

Analysis: Switching from vendored groundingdino/ to the groundingdino-py package

Date: 2026-07-29
Context: Task list in cleanup_groundingdino_directory_task_list.md proposes replacing the
vendored groundingdino/ source tree with the public groundingdino-py conda package.
This document records what the public package actually contains, how it diverges from the
vendored source, and the realistic approaches for adopting it.

---

TL;DR

The public groundingdino-py package is older and less capable than the groundingdino/
source currently vendored in this repo. Adopting it as-is is a downgrade that requires
patches to be re-applied on top just to run against this repo's pinned dependencies.

• Only version available: 0.4.0 (conda-forge and PyPI), published ~3 years ago.
  Its internal version.py even reads 0.3.0.
• Build: noarch / pyhd8ed1ab_0 — pure Python. No compiled _C.so, and no
  csrc/ sources either, so the CUDA op cannot be built from it.
• Uses only the pure-Python deformable-attention path (slower than the vendored CUDA _C kernel).
• bertwarper.py is unpatched and will very likely crash on this repo's pinned
  transformers==4.49.0.
• Hard-pins supervision 0.6.0 (repo's pixi/recipe request >=0.22.0).
• Bundles the upstream GroundingDINO_SwinB_cfg.py, not Azar's tuned config.

---

How this analysis was produced

1. Downloaded the actual package:
   https://conda.anaconda.org/conda-forge/noarch/groundingdino-py-0.4.0-pyhd8ed1ab_0.conda
2. Extracted it (.conda = zip → pkg-*.tar.zst) and listed contents.
3. Diffed every *.py in the package against groundingdino/ in this repo.
4. Cross-referenced the divergences against the repo's git history to attribute changes
   to Azar (Di Yao / Azhar Hasan) vs. upstream contributors.
5. Grepped all consumers (demo/, eval/, tracker/, ros2_package/) to determine which
   divergent APIs are actually used outside the groundingdino/ package itself.

---

Package facts (verified)

Property	Value
Latest version (conda-forge)	0.4.0 (only version)
Latest version (PyPI)	0.4.0 (releases: 0.1.0, 0.1.2, 0.2.0, 0.3.0, 0.4.0)
Age	~3 years old
Build string	pyhd8ed1ab_0 (noarch, pure Python)
Internal version.py	0.3.0 (mislabeled)
Compiled _C / .so	None
csrc/ CUDA sources	None
Bundled config files	Upstream GroundingDINO_SwinB_cfg.py, GroundingDINO_SwinT_OGC.py
Declared conda deps	addict, numpy, opencv, pycocotools, python>=3.7, pytorch, supervision 0.6.0, timm, torchvision, transformers, yapf

---

Differences between package 0.4.0 and vendored groundingdino/

Nine files differ. They fall into two categories by how they'd have to be handled if the
package were adopted.

Category A — data files (passed by path, never imported)

File	Difference	Consumer impact
config/GroundingDINO_SwinB_cfg.py	Azar heavily rewrote this (commits 3526444 "mot17 eval integrated", 3d14936 "results drafted") — added data_aug_*, tuned dn_labelbook_size, return_interm_indices, etc.	Every consumer passes groundingdino/config/GroundingDINO_SwinB_cfg.py as a filesystem path to load_model(). Azar's tuned config must be preserved.
config/GroundingDINO_SwinT_OGC.py	Minor / upstream.	Referenced by path.

Because configs are loaded by path (not imported), the package's bundled configs are
irrelevant — the repo just needs its own copies on disk at the referenced paths.

Category B — code inside the installed package

File	Difference (vendored has, package lacks)	Attribution	Used by external consumers?
models/GroundingDINO/bertwarper.py	transformers-compat fix: hasattr guard on get_head_mask; passes model dtype (not device) to get_extended_attention_mask	Azar (f119b8e "updated to use the newer Transformer API with breaking changes")	Indirectly — mandatory; unpatched package crashes on transformers==4.49.0
util/inference.py	(1) newer annotate using sv.BoxAnnotator + sv.LabelAnnotator (supervision ≥0.22 API); (2) remove_combined param on predict; (3) phrases2classes substring matching	upstream (ethanlee928 d75c95d, 8b6a55f) + upstream PRs	annotate only by demo/gradio_app.py; remove_combined & phrases2classes not used externally
util/utils.py	get_phrases_from_posmap gains left_idx/right_idx params	upstream (paired with remove_combined)	Not used externally
models/GroundingDINO/groundingdino.py	set_image_tensor / set_image_features / unset_image_tensor + feature-caching forward	upstream PR #160 "decoupled image processing"	Not used externally
datasets/cocogrounding_eval.py	Present in repo, entirely absent from package	upstream #149	Yes — demo/test_ap_on_coco.py
models/GroundingDINO/ms_deform_attn.py	Full CUDA _C path (MultiScaleDeformableAttnFunction, try: from groundingdino import _C); package has only the pure-Python multi_scale_deformable_attn_pytorch	upstream + Azar build fix (e8c92be)	Indirectly — cannot be overlaid (see below)
__init__.py / version.py	version strings only	—	none

Key filter: the repo's consumers (demo/, eval/, tracker/, ros2_package/) only call
load_model, predict, load_image, annotate, build_model, and datasets.transforms.
The newer-only internals (remove_combined, set_image_features, improved phrases2classes)
are used only inside groundingdino/ — so their absence in the package does not break
external callers.

Supervision resolves itself: supervision is imported only by groundingdino/util/inference.py
— nothing in the tracker/eval/demo imports it directly. So if you accept the package's
supervision 0.6.0 pin, the package's own annotate matches that API and works unmodified —
provided you do not force >=0.22.0.

---

Why the package breaks / regresses against this repo

1. transformers crash (hard). Repo pins transformers==4.49.0 (r.txt) / >=4.49.0 (pixi).
   The package's unpatched bertwarper.py calls get_extended_attention_mask(..., device) and
   assigns get_head_mask unconditionally — the exact breakage Azar's f119b8e fixed. Adopting
   the package requires re-applying that fix.
2. Loss of CUDA deformable attention (perf regression). The vendored code runs the fused
   _C kernel on GPU; the package only has the pure-Python path. Functionally OK, measurably
   slower for real-time tracking. Not patchable without recompiling (see below).
3. supervision pin conflict. 0.6.0 (package) vs >=0.22.0 (pixi/recipe). Avoidable only by
   accepting 0.6.0 (which is fine, since only the package's own inference.py uses it).
4. Config + missing modules. Azar's tuned SwinB config, csrc/, and
   datasets/cocogrounding_eval.py are not in the package and must be re-added on top.

---

The one thing that cannot be overlaid: _C

The package is noarch and ships neither the compiled _C.so nor the csrc/ sources.
To restore the fused CUDA kernel you'd have to bring csrc/ back from this repo and run a
compile step against torch — i.e., build from source, which is exactly what adopting the package
was meant to eliminate. Under "adopt the package," the pure-Python (slower) deformable attention
is a permanent, unpatchable regression.

---

Approaches

Approach 1 — Build your own conda package from recipe.yaml (recommended)

The repo already has recipe.yaml and a pixi conda-build task
(rattler-build build --recipe recipe.yaml ...) that builds groundingdino 0.1.0 with the
CUDA _C extension, Azar's config, and every patch already in place.

• Pros: hits the real goal ("install as a package, not a live source tree") with zero
  overlay machinery; keeps _C (full speed), keeps transformers fix, keeps tuned config,
  keeps supervision >=0.22.
• Cons: the groundingdino/ source stays in the repo as build input (it's no longer
  imported in place, but it isn't deleted either). Requires the CUDA build toolchain at
  package-build time.
• Net: cleanest path; nothing is lost.

Approach 2 — Adopt public groundingdino-py 0.4.0 + overlay Azar's fixes

Depend on the conda-forge package, then re-apply what's needed:

Required steps

1. Keep Azar's config/*_cfg.py in-repo as data files and point consumers at them (trivial —
   they're already referenced by path).
2. Vendor datasets/cocogrounding_eval.py alongside (only needed for demo/test_ap_on_coco.py).
3. Ship one code patch — bertwarper.py transformers-compat — via one of:
   • Post-install patch (most deterministic): a .patch applied to site-packages right
     after pip install, as a pixi task and a Dockerfile RUN. Deterministic because there's
     exactly one frozen version (0.4.0) to target. Must re-run on every fresh env/image build.

     SITE=$(python -c "import groundingdino,pathlib;print(pathlib.Path(groundingdino.__file__).parent)")
     patch -p1 -d "$SITE" < patches/bertwarper-transformers-compat.patch

   • Runtime monkeypatch (no touching site-packages): a groundingdino_patches.py imported
     once at startup. Clean for the __init__ attribute tweak, but the forward() dtype fix
     means replacing the whole ~90-line method — copy-paste that rots against upstream.

     import groundingdino.models.GroundingDINO.bertwarper as bw
     _orig_init = bw.BertModelWarper.__init__
     def _patched_init(self, bert_model):
         _orig_init(self, bert_model)
         if not hasattr(bert_model, "get_head_mask"):
             self.get_head_mask = lambda x, y: None
     bw.BertModelWarper.__init__ = _patched_init
     # forward() dtype fix still requires overriding the full method

4. Accept supervision 0.6.0 (don't force >=0.22.0).
5. Accept pure-Python deformable attention (permanent perf regression).

Optional (only if you want feature parity with the vendored code): also overlay
util/inference.py (remove_combined, phrases2classes), util/utils.py, and
groundingdino.py (feature caching) — but no current consumer needs them.

• Pros: matches the task literally; removes most of the source tree.
• Cons: maintains a post-install patch against a frozen 3-year-old package; loses the CUDA
  kernel; extra fragility on every rebuild.

Approach 3 — Keep vendored source, do the other cleanups only

Don't switch to any package. Keep groundingdino/ built via setup.py, but still consolidate
configuration: merge pixi.toml into pyproject.toml, reconcile r.txt vs the
requirements.txt referenced by README.md / Dockerfile.ros2 / environment.yaml, and tidy
the Dockerfiles.

• Pros: nothing regresses; still addresses the config-sprawl part of the task list.
• Cons: does not achieve "use the package instead of source."

Approach 4 — Build groundingdino-py from upstream GitHub main (not the 0.4.0 release)

Upstream's current setup.py does compile _C and is far newer than the 0.4.0 conda
release. Building from a pinned upstream commit (or a fork) would give a modern, CUDA-accelerated
package — but still requires re-applying Azar's config + bertwarper fix and a compile step,
making it functionally similar to Approach 1 with an external source of truth.

---

Recommendation

Approach 1 (build your own conda package from recipe.yaml) achieves the task's real intent
— installing GroundingDINO as a package rather than importing a live source tree — while keeping
the CUDA _C kernel, Azar's tuned config, and the transformers fix, with no overlay
machinery. Approach 2 (public 0.4.0) is viable but strictly a downgrade requiring a maintained
patch and eating the CUDA perf loss.

---

Appendix — files that MUST be preserved regardless of approach

• groundingdino/config/GroundingDINO_SwinB_cfg.py — Azar's tuned config (data file, by path).
• groundingdino/config/GroundingDINO_SwinT_OGC.py — referenced by path.
• groundingdino/models/GroundingDINO/bertwarper.py — Azar's transformers-compat fix.
• groundingdino/models/GroundingDINO/csrc/** + ms_deform_attn.py CUDA path — only if the
  fused _C kernel (full speed) is to be retained.
• groundingdino/datasets/cocogrounding_eval.py — required by demo/test_ap_on_coco.py.

[phreed]

Analysis: Does the ROS2 container need a GPU, and how to run it on CPU

Subject: docker/Dockerfile.ros2 (image groundingdino_ros) in selectivetracking.
Question: Does the container require a GPU to run, and can it run CPU-only?

---

TL;DR

• Build: needs no GPU and no CUDA toolkit on the host — the CUDA op is cross-compiled ahead-of-time.
• Run (as shipped): requires a GPU. The ROS2 node hardcodes device='cuda' with no
  availability check, so on a GPU-less host it crashes at model load.
• CPU is possible but not out-of-the-box: override the ROS params
  device:=cpu use_fp16:=false. It will be slow, and use_fp16 must be off
  because a core CPU op (grid_sample) has no fp16 implementation.

---

Build vs. run — only run needs a GPU

Phase	GPU needed?	Host needs
Build (build_ros2.sh / CI)	No	Docker only. The image installs its own CUDA toolkit; Dockerfile.ros2 sets FORCE_CUDA=1 + TORCH_CUDA_ARCH_LIST, so nvcc compiles the _C op AOT for the listed architectures without a GPU present. (A plain docker build can't access GPUs anyway.)
Run (default launch)	Yes	NVIDIA driver + nvidia-container-toolkit; docker run --gpus all. The host needs only the driver, not the full CUDA toolkit (the runtime libs ship in the image).

So a CI stage that only builds/pushes the image needs no GPU; only a stage that
actually runs inference does.

---

Why the container requires a GPU by default

The default CMD is ros2 launch groundingdino_ros groundingdino.launch.py,
which starts the node. The device is hardcoded to CUDA with no is_available()
guard:

1. ros2_package/groundingdino_ros/groundingdino_ros/groundingdino_node.py:104
   → declare_parameter('device', 'cuda'), :105 → use_fp16=True.
2. ros2_package/groundingdino_ros/config/default_params.yaml:6 → device: "cuda".
3. The node passes device='cuda' into eval/worker_simple.py's Worker (:160-161).
4. eval/worker_simple.py:156:
   self.device = device if device else ("cuda" if torch.cuda.is_available() else "cpu")
   — because the node passes a truthy "cuda", the is_available() fallback is
   skipped.
5. eval/worker_simple.py:161 → self.dino_model = self.dino_model.to(self.device) → .to("cuda").
   (:214 t.cuda(...), :215 .half(), :220 autocast are all CUDA-bound too.)

Reproduced on a GPU-less host (CUDA_VISIBLE_DEVICES=""):

torch.cuda.is_available(): False
.to("cuda") -> RuntimeError: No CUDA GPUs are available

The entrypoint (docker/ros2_entrypoint.sh) only prints
Warning: CUDA not available and does not abort — so the container starts,
then crashes at Worker construction.

Conclusion: the GPU requirement is a configuration default (hardcoded
device='cuda', no availability guard), not an intrinsic limitation.

---

How to run it CPU-only

The code is CPU-capable if you stop forcing CUDA:

ros2 launch groundingdino_ros groundingdino.launch.py device:=cpu use_fp16:=false

Why this works:

• eval/worker_simple.py:156 auto-selects CPU when device is not forced.
• The model runs on CPU: groundingdino/models/GroundingDINO/ms_deform_attn.py
  takes the pure-Python multi_scale_deformable_attn_pytorch path when not on
  CUDA, so the compiled _C CUDA op is not required.

For a Docker run without --gpus, also drop the GPU flags and pass the CPU
params (e.g. via the launch args / a params override). Expect substantially
slower inference — CPU has no fused deformable-attention kernel.

---

Why use_fp16 must be false on CPU

Two independent reasons:

1. Missing kernel → hard crash. The CPU deformable-attention fallback uses
   F.grid_sample (ms_deform_attn.py:118), which has no fp16 CPU
   implementation. On torch 2.4.0 (this repo's version) it raises:

   RuntimeError: grid_sampler_2d_cpu not implemented for Half
   

   (Many generic ops — conv, linear, softmax — do have fp16 CPU kernels in
   torch 2.4, but grid_sample, the one this model needs on CPU, does not.)
2. No speedup anyway. x86 CPUs have no native fp16 compute; where fp16 CPU
   kernels exist they up-convert to fp32 internally, so fp16 is slower, not
   faster. PyTorch's CPU mixed precision path is bfloat16
   (torch.autocast(device_type="cpu")), not fp16 — and this code uses
   torch.cuda.amp.autocast, which is inert on CPU tensors.

So on CPU you must keep everything fp32 (use_fp16:=false), which has full CPU
kernel coverage.

---

Summary table

Scenario	Works?	Notes
Build the image without a GPU	✅	AOT CUDA compile; host needs only Docker
Run default (device=cuda) on a GPU host	✅	needs --gpus all + driver + nvidia-container-toolkit
Run default on a GPU-less host	❌	crashes: RuntimeError: No CUDA GPUs are available
Run device:=cpu use_fp16:=false	✅ (slow)	pure-Python deformable attention; _C not needed
Run device:=cpu use_fp16:=true	❌	grid_sampler_2d_cpu not implemented for Half

[phreed]

Status: all four objectives done, pending upstream review

The four fixes are split into separate pull requests against upstream, the feedstock builds
them, and this repository now consumes the result. Verified end to end.

Pull requests

Against IDEA-Research/GroundingDINO:

PR	Branch	Change	Kind
#458	fix/transformers-api-compat	bertwarper.py, +11/−2	fix
#459	fix/cuda-fp16-include	ms_deform_attn_cuda.cu, +2/−1	fix
#460	feat/version-attribute	version.py, +1	packaging
#461	feat/swinb-training-config	GroundingDINO_SwinB_cfg.py, +231/−3	proposal

Those four files are the complete set of changes this repository made under groundingdino/
since it forked. The fork point is 856dde2 ("Grounded SAM 2 Release"), which is also upstream's
current HEAD, so nothing had to be rebased. Eighteen months of work, four files — everything
else was always this project's own research.

#461 is deliberately framed as a proposal rather than a fix: it changes an inference config into
one that also describes training, which changes the meaning of a file other people read. It offers
a separate GroundingDINO_SwinB_train_cfg.py as an alternative if upstream prefers that.

Branches not yet proposed

• phreed/groundingdino-py-feedstock@feat/rattler-build-multi-output — converts the feedstock
  from meta.yaml/conda-build to recipe.yaml/rattler-build and splits it into per-backend
  outputs. Not yet a PR to conda-forge, because the recipe currently builds
  phreed/GroundingDINO@integration/split-fixes (a branch merging all four PRs above). conda-forge
  will not take a recipe pointing at a personal fork; this repoints at an upstream tag once
  #458–#461 land. Relates to Active Maintainer conda-forge/groundingdino-py-feedstock#2.
• phreed/selectivetracking@feat/use-groundingdino-package — this repository's side of the
  change. Not yet a PR; say the word and I will open one.

Re: the "adopting groundingdino-py is a downgrade" analysis above

That analysis was correct about groundingdino-py 0.4.0, and every objection it raised is
what the new build had to fix. Point by point, against the artefacts actually produced:

0.4.0 objection	0.4.1 as built
Only 0.4.0, ~3 years old	0.4.1, built from the integration branch
noarch pure Python, no compiled _C.so	groundingdino-py-cuda ships _C.cpython-3X-x86_64-linux-gnu.so, compiled AOT for 8.0;8.6;8.9
No csrc/, so the CUDA op cannot be built	Correct, and no longer relevant — the op ships prebuilt rather than needing local compilation
Only the slow pure-Python deformable-attention path	The compiled kernel is used when present; the Python path remains the documented fallback
bertwarper.py unpatched, crashes on transformers==4.49.0	Carries #458; verified hasattr guard and dtype fix present in the installed package
Hard-pins supervision 0.6.0	supervision >=0.22.0
Bundles upstream GroundingDINO_SwinB_cfg.py, not the tuned one	Bundles the tuned config; lr_backbone_names and the rest of the training block are present

So the shape of the answer is: don't adopt 0.4.0, adopt a package built from the fixed source.
That is what the feedstock branch does.

Packages produced

groundingdino-py-base   0.4.1  pyh7511447_0            pure Python, noarch
groundingdino-py-cuda   0.4.1  cuda121_py310_...       the compiled extension, one file
groundingdino-py-cuda   0.4.1  cuda121_py311_...
groundingdino-py-cuda   0.4.1  cuda121_py312_...
groundingdino-py-cpu    0.4.1  cpu_0                   metapackage constraining _C out
groundingdino-py        0.4.1  h4616a5c_0              umbrella

The split is along the only axis that exists in the source: whether the extension is present.
ms_deform_attn.py already guards from groundingdino import _C in a try/except, so the
base package is correct without it — merely slow, and confined to float32, since grid_sample
has no half-precision CPU kernel. That last point is the same one the ROS2 CPU/GPU analysis above
reaches from the other direction.

There is deliberately no rocm or vulkan output. The only HIP reference in GroundingDINO is
a dead branch in setup.py that returns None before its macro reaches a compiler, and there is
no Vulkan code at all; both would have shipped the pure-Python fallback under a name promising
acceleration. A real ROCm output needs the CUDA kernel hipified upstream.

Verified end to end

In this repository's environment, resolving against the built channel:

resolves to : .../st/.pixi/envs/default/lib/python3.10/site-packages/groundingdino/__init__.py
extension   : ['..._C.cpython-310-x86_64-linux-gnu.so']
_C forward  : True
local config: groundingdino | training params: True

groundingdino resolves to the installed package rather than to the ./groundingdino directory
left in the working copy. That works because only groundingdino/config/ survives the purge and
groundingdino/__init__.py is deleted: a directory without __init__.py is merely a namespace
portion, and a namespace portion loses to a regular package found later on sys.path. Left in
place it would have shadowed the dependency silently. docker/Dockerfile.ros2 now asserts this at
image build time rather than leaving it to be discovered at run time.

groundingdino/config/ is kept because 20 call sites across 18 files load those two files by
path, not by import.

Known nit

version.py reads 0.1.0 while the package is 0.4.1 — the same mislabelling the analysis above
noted in 0.4.0 (whose version.py read 0.3.0). #460 only adds the attribute; making it agree
with the release version is a separate question for upstream, who own the numbering.

Blocked on

#458–#461 merging. Then the feedstock recipe and Dockerfile.ros2 both repoint from
phreed:integration/split-fixes to an upstream tag, and the feedstock change can go to
conda-forge.

[phreed]

The two remaining branches now have pull requests, both opened as drafts:

PR	Repo	What
phreed/selectivetracking#2	this one	Removes the vendored tree, depends on the package, merges pixi.toml into pyproject.toml
conda-forge/groundingdino-py-feedstock#3	feedstock	meta.yaml → recipe.yaml, rattler-build, per-backend outputs. Addresses conda-forge/groundingdino-py-feedstock#2

Both are drafts on purpose, and for the same reason: each still points at
phreed:integration/split-fixes, the branch merging IDEA-Research/GroundingDINO#458–#461. The
feedstock cannot build IDEA-Research/main until #459 lands — without the cuda_fp16.h include
the CUDA extension does not compile under CUDA 12 at all, so there would be nothing for the
-cuda output to ship.

Full set, in dependency order:

1. fix: restore compatibility with transformers >= 4.49 IDEA-Research/GroundingDINO#458, #459, #460 — the fixes packaging needs
2. Convert to recipe.yaml and split into per-backend outputs conda-forge/groundingdino-py-feedstock#3 — repoint at an upstream tag, undraft, merge
3. feat: depend on the groundingdino conda package instead of vendoring it #2 — drop the local channel, undraft, merge
4. feat: extend the SwinB config with training parameters IDEA-Research/GroundingDINO#461 — independent of the above; can land whenever

[phreed]

The feedstock no longer waits on upstream review. Per @phreed's suggestion, the three fixes are
carried as patches rather than by pinning the recipe at a fork branch — which is conda-forge's
mechanism for exactly this situation, and unblocks everything downstream.

conda-forge/groundingdino-py-feedstock#3 is now out of draft.

What changed

	Before	Now
Source	phreed/GroundingDINO@integration/split-fixes	IDEA-Research/GroundingDINO at 856dde2, pinned by sha256
Fixes	baked into a fork branch	recipe/patches/0001..0003, generated with git format-patch from the same branches
Mergeable	no — conda-forge will not take a personal fork	yes

A feedstock has to build from a source anyone can fetch and verify. The fork pin failed that, and
made every downstream step queue behind upstream review for no reason.

Which patches, and which not

• 0001 → fix: restore compatibility with transformers >= 4.49 IDEA-Research/GroundingDINO#458 — transformers >= 4.49 compatibility
• 0002 → fix: include cuda_fp16.h in the MsDeformAttn CUDA kernel IDEA-Research/GroundingDINO#459 — cuda_fp16.h; without it the extension does not
  compile under CUDA 12 at all, so -cuda would have nothing to ship
• 0003 → feat: expose __version__ on the groundingdino package IDEA-Research/GroundingDINO#460 — __version__, which packaging has to read

IDEA-Research/GroundingDINO#461 is deliberately not carried. It extends the SwinB config with
training parameters — a proposal rather than a fix — and no consumer needs it from the package,
because the configs are read from disk by path. This repository keeps its own tuned copy in
groundingdino/config/ and loads that, so nothing here regresses. A feedstock carrying a
contested upstream feature is a fork wearing a different hat.

Each patch is deleted as its pull request lands.

Verified

The patches apply cleanly to a pristine upstream tree and reproduce this repository's versions of
all three files byte-for-byte. Rebuilt from upstream + patches and compared against the earlier
fork-pinned build: identical file sets in every output. The only content differences are the
SwinB config (intended — #461 excluded), the dist-info RECORD and direct_url.json (the source
URL changed), and the compiled .so, which is not byte-reproducible.

Revised order

1. Convert to recipe.yaml and split into per-backend outputs conda-forge/groundingdino-py-feedstock#3 — ready for review now, merges independently
2. feat: depend on the groundingdino conda package instead of vendoring it #2 — drop the local channel once the package publishes, undraft, merge
3. fix: restore compatibility with transformers >= 4.49 IDEA-Research/GroundingDINO#458, #459, #460 — land on their own schedule; drop each patch as it does
4. feat: extend the SwinB config with training parameters IDEA-Research/GroundingDINO#461 — independent of all of the above

Only step 2 now depends on anything else.

[phreed]

The feedstock recipe is down to two outputs from four, on @phreed's observation that the
metapackages were redundant.

	Before	Now
groundingdino-py	empty metapackage depending on -base	the detector — 42 files, noarch
groundingdino-py-base	held every file	folded into the above
groundingdino-py-cpu	empty metapackage + a constraint	removed
groundingdino-py-cuda	the compiled _C	unchanged

The observation was right, though the redundancy ran the other way round from how it looked:
-base was the one carrying all the content, and groundingdino-py was the empty shim in front
of it. So rather than deleting -base, the fix is to name it groundingdino-py and delete the
shim. Installing that name still gets the detector, exactly as before.

-cpu is gone too. It was the same empty metapackage plus constrains: groundingdino-py-cuda <0.0a0, guarding against something that cannot happen by accident — -cuda is opt-in and nothing
depends on it, so not installing it is how you do not install it.

Verified against the artefacts, not by inspection

The new groundingdino-py carries the same 42 files as the old -base, the same dependency list,
and the same noarch: python type. The only content differences are
dist-info/direct_url.json, which records the build directory whose name changed with the output,
and RECORD, which hashes it. -cuda is still exactly _C.cpython-*.so and still shares no path
with the package it depends on.

conda-smithy lint stays clean, and this repository still installs, imports groundingdino from
site-packages, and loads _C.

One knock-on here

pyproject.toml advised swapping groundingdino-py-cuda for groundingdino-py-cpu on a machine
without a GPU. There is no -cpu any more, and the correct advice is simpler: drop the -cuda
line. Fixed in #2.

[phreed]

What it takes to get a working CUDA package — two things, one of which was a real bug nobody
had hit yet.

1. A fourth upstream fix: the kernel does not compile under CUDA 12.9

Every local build so far used CUDA 12.1, and that hid a hard failure. conda-forge's current
pinning is 12.9, where the extension does not compile at all:

ms_deform_attn_cuda.cu(66): error: no suitable conversion function from
"const at::DeprecatedTypeProperties" to "torch::headeronly::ScalarType" exists

AT_DISPATCH_FLOATING_TYPES takes a ScalarType. Tensor::type() returns
at::DeprecatedTypeProperties, which used to convert implicitly and no longer does.
value.scalar_type() is the replacement, and value.type().is_cuda() is value.is_cuda() for the
same reason — fifteen call sites across ms_deform_attn.h and ms_deform_attn_cuda.cu.

Submitted as IDEA-Research/GroundingDINO#462, and carried in the feedstock as patch 0004
until it merges. Verified both ways: 12.1 and 12.9 now each produce a working
_C.cpython-310-x86_64-linux-gnu.so.

This is worth noting beyond this issue: anyone building GroundingDINO's CUDA op against a recent
toolchain hits it, which is exactly the kind of thing that makes people republish a patched fork
instead of upstreaming.

2. A re-render on conda-forge's infrastructure

The feedstock's CI variant still says cuda_compiler_version: [None], so CI would not build
-cuda even with the code fixed.

This should be automatic rather than something to configure. The global pinning gates the CUDA row
on an environment variable:

cuda_compiler_version:
  - None
  - 12.9    # [... and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

and conda-smithy sets CF_CUDA_ENABLED when it finds compiler('cuda') in the recipe — which
ours has. Evaluating the pinning by hand with it set does give [None, 12.9], with the three keys
it is zipped against each growing to two entries so the zip stays balanced.

My local re-render did not reproduce that — in the same run rattler-build rendered cuda129
variants while the .ci_support writer emitted None — which points at tooling on this machine
rather than the recipe. So the ask on conda-forge/groundingdino-py-feedstock#3 is for a maintainer
to run @conda-forge-admin, please rerender and confirm the variant comes out with 12.9.

I did try hand-writing the pinning: recipe/conda_build_config.yaml with only
cuda_compiler_version is silently dropped on a zip-length mismatch, and extending the whole zip
means setting c_stdlib_version, which fails linting against the macOS baseline. Both verified,
then reverted — it is the wrong layer to fix this at.

Current PR set

PR	State
IDEA-Research/GroundingDINO#458	transformers >= 4.49 compatibility
IDEA-Research/GroundingDINO#459	cuda_fp16.h, fixes the CUDA 12 build
IDEA-Research/GroundingDINO#460	__version__
IDEA-Research/GroundingDINO#461	SwinB training config (proposal)
IDEA-Research/GroundingDINO#462	Tensor::type(), fixes the CUDA 12.9 build
conda-forge/groundingdino-py-feedstock#3	carries 458/459/460/462 as patches
#2	consumes the package