[BUG]: repository too large

[aterrel]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct

Describe the bug

git clone for local development requires 650MB download. This is not something that one can do at conferences with shotty internet where the material is presented.

Steps to reproduce

git clone

Expected behavior

faster download

Actual behavior

very slow download

Environment

No response

Additional context

No response

[robobryce]

I dug into where the size actually comes from. Summary: the checked-out tree is tiny (~10 MB) — essentially all of the download is Git history and non-main branches.

Measurements

What	Blob bytes
Working tree of main (668 files)	~10 MB
History reachable from main only	163.5 MB
History reachable from all 27 branches	769.7 MB

So roughly 600 MB lives on branches other than main, and a default git clone fetches all of them. The heavy non-main refs include event branches (2025-09-ndc-techtown, …), library branches (cuDF, cuGraph, cuML, new-cugraph, fixed-cugraph), the generated CI-artifact branch, and 7 pull-request/* copies.

On top of that, main's own history carries ~150 MB of large blobs that have since been deleted or relocated — e.g.:

• Accelerated_Python_User_Guide/notebooks/*NavierStokes*In_Warp.ipynb (~70 MB each, multiple revisions — notebooks with embedded output)
• gpu-cpp-tutorial/slides/ACH-0{1,2,3}*.pdf (25–56 MB each)
• tutorials/gpu-python-tutorial/data/pageviews_small.csv (~32 MB)

These were committed directly into Git rather than through LFS. The current .gitattributes does route *.pdf/*.png/*.pptx/... through LFS (today's binaries are proper LFS pointers), so the problem is historical/legacy, not ongoing.

Immediate workaround (no repo changes, good for workshops)

A shallow single-branch clone drops the .git directory dramatically (measured ~3 MB vs the full clone):

git clone --depth 1 --single-branch https://github.com/NVIDIA/accelerated-computing-hub.git

Worth adding to the README/CONTRIBUTING for conference attendees. --filter=blob:none (blobless partial clone) is another option that fetches blobs on demand.

Longer-term options (maintainer decisions — no single PR fixes clone size)

1. Prune stale branches — deleting merged event/library/PR-copy branches removes them from the default fetch. Lowest-risk, biggest single win since ~600 MB is off-main.
2. History rewrite (e.g. git filter-repo) to purge the deleted large blobs from main's history. Most thorough, but it's a coordinated, force-push operation that rewrites SHAs and breaks existing forks/PRs — a repo-admin call.

Happy to prep whichever direction you prefer — e.g. a docs PR for the shallow-clone instructions (1), or a branch-cleanup plan / filter-repo runbook for (2). I held off opening a PR since the right fix depends on which path you want.

[robobryce]

Here's the per-branch breakdown to inform pruning. Important caveat on the numbers: the "unique vs main" sizes are not additive — the big branches share most of their blobs. The three event branches each show ~430 MB alone but total 430 MB combined (near-total overlap), so they only pay off if removed as a group.

✅ Safe to prune (stale, superseded, low risk)

Branch	Last commit	Notes
cuDF	2024-10-28	Old per-library branches (Melody Wang), untouched ~1.5 yrs,
cuGraph	2024-10-15	superseded by current tutorial layout. Combined unique
cuML	2024-10-29	footprint of all five ≈ 66 MB.
fixed-cugraph	2024-11-03	fixed-cugraph / new-cugraph look like one-off
new-cugraph	2024-11-01	working branches.
pull-request/180	2026-06-08	copy-pr-bot branch for merged PR #180 — safe to delete.

⚠️ Biggest footprint, but policy-protected — your call

Branch	Combined unique	Why I'm flagging, not recommending
2025-09-flatiron-institute-cuda-python, 2025-09-ndc-techtown, 2025-10-cpp-under-the-sea	≈430 MB (shared)	CONTRIBUTING.md §Branches explicitly designates these as intentional long-lived event archives ("capture the exact state of tutorials as delivered at a specific event"). Pruning them is the single biggest win but contradicts that stated policy — a maintainer decision. If the policy still holds, an alternative is moving them to tags or a separate archive repo.
cuda-python-dli	~55 MB	Looks like a DLI course branch (last commit 2025-12-11). Keep if the course is live.

❌ Do NOT prune (active / managed)

• generated — CI-managed artifact branch (auto-recreated; CONTRIBUTING.md §generated).
• pull-request/{124,125,138,152,161,162} — all map to OPEN PRs (e.g. 2-D Navier-Stokes Solver in a periodic box  #124 is the 280 MB Navier-Stokes solver still under review). copy-pr-bot manages these; they clear when the PRs close/merge.
• dependabot/pip/.../pip-5563803c48 — backs open PR Bump jupyterlab from 4.5.4 to 4.5.7 in /tutorials/accelerated-python/brev in the pip group across 1 directory #175.
• cbrower/floating-point-emulation — backs open PR Floating Point Emulation Tutorial #125.
• Small feature/test branches (feature/*, test/*, cuda-tools-tests) — ~0 MB each, no meaningful size impact.

Bottom line

The cugraph/lib group (~66 MB) is the clean, no-controversy win. The ~430 MB of event branches dwarfs everything else but is protected by current branch policy — if reducing clone size is the priority, that's the conversation worth having (archive-as-tags, or a --single-branch clone recommendation in the meantime, since these branches aren't fetched when you clone only main).

[brycelelbach]

Instead of pruning the event branches, could we rewrite them to use git lfs correctly and purge binary objects from them?

[robobryce]

Yes — that's a better option than deletion, since it keeps the archives intact while removing them as a clone-size burden. I dug into the specifics so we know exactly what a rewrite would involve.

What's actually in the ~430 MB that's unique to the three event branches

Content	Size	LFS-tracked today?
PDF / PNG / GIF / JPG (binaries)	317 MB	✅ yes (.gitattributes patterns)
.ipynb notebooks	70 MB	❌ no — and shouldn't be
.csv data	33 MB	❌ no
everything else	~11 MB	n/a

Two things this surfaces:

1. The event branches have no LFS config at all — their .gitattributes has no filter=lfs lines, and the PDFs are stored raw (%PDF-1.3..., not pointers). So they predate the LFS setup entirely. A rewrite would need to both add the LFS attributes and convert the existing blobs (e.g. git lfs migrate import --everything --include="*.pdf,*.png,*.gif,*.jpg,*.pptx").

2. Migrating to LFS only covers 317 of the 430 MB. The other ~100 MB is notebooks and CSVs:

   • The 70 MB of .ipynb is embedded cell output, not the notebooks themselves. Putting .ipynb in LFS is a bad idea (breaks GitHub rendering and diffs) — the right fix there is stripping outputs (nbconvert --clear-output / nbstripout), which the repo's own notebook-format hook philosophy already leans toward.
   • The 33 MB CSV (pageviews_small.csv) isn't covered by the current LFS patterns; we'd add *.csv (or just that path).

Honest caveat on the clone-size payoff

A default git clone still pulls the LFS objects for the branch it checks out — LFS shrinks the Git pack, not necessarily the bytes-on-the-wire for someone who checks out an event branch. The real win is:

• The event blobs stop riding along in the shared packfile, so cloning main (or --single-branch anything) no longer drags them in. Since main is already clean (~10 MB), this is mostly about keeping the full clone honest.
• LFS objects can be fetched lazily / skipped (GIT_LFS_SKIP_SMUDGE=1, --filter=blob:none), which a raw-blob history doesn't allow.

Caveat that applies to any rewrite

Rewriting these branches changes their commit SHAs and requires a force-push. Anyone with the old event branches checked out (or links pinned to old commit SHAs) is affected. Since these are archival, that's likely fine — but worth a heads-up in the commit/PR.

Proposed approach

I can put together a git lfs migrate + output-strip runbook (exact commands, per branch, with before/after size verification) for you to run with repo-admin rights — I can't force-push to upstream myself. Want me to:

• (a) draft that runbook here, and/or
• (b) prototype it on my fork's copies of the three branches and report the measured before/after .git sizes so we know the real payoff before you touch upstream?

I'd lean toward (b) first — measure, then commit to the rewrite.

[robobryce]

Measured it. The headline: a full clone is ~438 MB packed, of which main alone is only 42 MB — the event branches are the bulk of it. I used packed (on-the-wire) sizes throughout, since that's what a clone actually transfers; raw byte counts overstate the win because PDFs are already compressed while notebooks/CSVs shrink a lot.

How much an LFS-rewrite of the event branches frees

Scenario	Packed freed
Rewrite the 3 event branches	~277 MB
Rewrite the 5 cugraph/lib branches alone	~0 MB
Rewrite event + cugraph/lib together	~327 MB

The non-obvious part: the cugraph/lib branches free nothing on their own, because every large blob they hold is also carried by an event branch (the ACH slide PDFs). Conversely, rewriting the event branches frees 277 MB, but ~50 MB of their content survives until the lib branches are cleaned too. So the two groups overlap — to actually reclaim the full ~327 MB you have to rewrite both sets in the same pass.

What "rewrite" has to cover to hit that 327 MB

The 327 MB packed breaks down (raw → packed):

Content	Raw	Packed	Handling
PDF/PNG/GIF/JPG	317 MB	286 MB	git lfs migrate import — already-compressed, so packed≈raw
.ipynb	70 MB	18 MB	strip outputs (not LFS) — base64 output compresses ~4×
.csv	33 MB	17 MB	add *.csv to LFS patterns
other	11 MB	5 MB	—

So ~286 MB of the win is the binary→LFS migration; the remaining ~40 MB packed needs notebook-output stripping + adding CSV to LFS (neither is covered by the current .gitattributes).

Bottom line

• Realistic clone-size reduction: ~327 MB packed (~440 MB → ~110 MB) if event + lib branches are rewritten together with binaries→LFS and notebook outputs stripped.
• Binary-only LFS migration (no notebook strip) gets you ~286 of that.
• And to be precise about LFS: this removes the blobs from the Git history pack (so clones/fetches of other branches stop dragging them along). Someone who actually checks out a rewritten event branch still downloads its LFS objects on checkout — unless they use GIT_LFS_SKIP_SMUDGE=1 / lazy fetch, which raw-blob history doesn't allow at all.

Happy to prototype this on my fork's copies of all eight branches and post the real before/after .git sizes to confirm the ~110 MB target before you run it on upstream.

[robobryce]

Update on this — here's what we did and what to expect.

What we did

1. Backed up and deleted the 8 large stale branches (the three event archives + cuDF/cuGraph/cuML/fixed-cugraph/new-cugraph). They held ~325 MB of binary history unique to them. Full backups are preserved at robobryce/accelerated-computing-hub (and an offline bundle), so nothing is lost.
2. Filed a GitHub Support ticket to run server-side garbage collection, since deleting branches alone doesn't reclaim the space — the unreachable objects sit in the packfile until GitHub repacks. (There's no documented GC schedule, so a Support request is the reliable way to trigger it.)

Anticipated sizes once GC completes

I measured these against the current repo state:

Clone	.git size	Notes
Recommended for workshops — --depth 1 --single-branch + skip LFS smudge	~5 MB (19 MB checkout)	Command below. Images fetched on demand.
Normal clone of main (default git)	~590 MB	~40 MB git history + ~547 MB of LFS images that live on main itself
Full clone, all branches, post-GC	~165 MB git history + LFS	down from ~440 MB; ~105 MB of the remainder is open PR #124

The honest takeaway

Deleting the stale branches shrinks the full clone's git history (~440 MB → ~165 MB) and reclaims GitHub-side storage, but it does not shrink a normal clone of main — because the bulk of what a default clone now downloads is the ~547 MB of LFS image objects referenced by main itself, plus the open PR #124's notebooks (~105 MB), neither of which branch deletion touches.

So the most effective thing for conference attendees is the clone command, not the branch cleanup:

GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 --single-branch \
  https://github.com/NVIDIA/accelerated-computing-hub.git

That's a ~5 MB download with all notebooks/code present; images materialize on demand via git lfs pull (or per-directory) only if needed.

If we want the default git clone to be small too, the next lever is reducing main's own LFS image footprint (resizing/consolidating images) — happy to scope that separately if it's worth pursuing.

[robobryce]

Here are the clone workarounds, all measured against the current repo. These help today — no waiting on GC — and are the most effective fix for the conference/poor-internet case.

Option 1 — Lightest: shallow + single-branch, skip LFS images

GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 --single-branch \
  https://github.com/NVIDIA/accelerated-computing-hub.git

~19 MB checkout (~5 MB .git). All notebooks and code are present; images come down as small LFS pointer files instead of the real bytes.

Option 2 — Skip LFS images, keep full history

GIT_LFS_SKIP_SMUDGE=1 git clone --single-branch \
  https://github.com/NVIDIA/accelerated-computing-hub.git

~56 MB checkout (~42 MB .git). Use if you want the full commit history of main but not the ~547 MB of images.

Option 3 — Blobless partial clone (fetches on demand automatically)

git clone --filter=blob:none --single-branch \
  https://github.com/NVIDIA/accelerated-computing-hub.git

~20 MB checkout (~6 MB .git). Git fetches file contents lazily the first time you open them — no manual step, but needs connectivity when you do.

Fetching images only when you need them

After an GIT_LFS_SKIP_SMUDGE=1 clone (Options 1–2), materialize images for just the tutorial you're presenting:

git lfs pull --include "tutorials/accelerated-python/**"

Verified: this pulls only that subtree's images (a few MB) and leaves everything else as pointers. To grab everything later: git lfs pull.

Note on GIT_LFS_SKIP_SMUDGE

It's an environment variable for that one git clone. On Windows PowerShell use $env:GIT_LFS_SKIP_SMUDGE=1; git clone .... You can also make it permanent for a clone with git -c filter.lfs.smudge= -c filter.lfs.process= clone ....

These could go in the README/CONTRIBUTING under a "Cloning for workshops / low-bandwidth" heading — happy to open that docs PR if useful.

[robobryce]

One more angle, @aterrel: for the conference scenario, you may not want to run git clone by hand at all.

If you have a coding agent handy (Claude Code, etc.), just ask it something like "clone the NVIDIA Accelerated Computing Hub repo" and let it do the work. A capable agent will typically notice it's a large/LFS-heavy repo and reach for a shallow, single-branch, or skip-smudge clone on its own — so you get the ~5–20 MB download without having to remember the exact GIT_LFS_SKIP_SMUDGE=1 --depth 1 --single-branch incantation.

If you want to make that even more reliable, a short note in the README (or a CLAUDE.md/agent hint) like "this repo is large; prefer a shallow single-branch clone with LFS smudge skipped" nudges the agent to do the right thing every time. Happy to add that alongside the workshop clone instructions if it's useful.

[aterrel]

no there is no reason to make it this large. We should have better hygiene than this. If the ACH has just become an dumping place for junk, then it is less valuable as a resource.

…

On Thu, Jun 11, 2026 at 12:41 AM RoboBryce ***@***.***> wrote: *robobryce* left a comment (NVIDIA/accelerated-computing-hub#184) <#184 (comment)> One more angle, @aterrel <https://github.com/aterrel>: for the conference scenario, you may not want to run git clone by hand at all. If you have a coding agent handy (Claude Code, etc.), just ask it something like *"clone the NVIDIA Accelerated Computing Hub repo"* and let it do the work. A capable agent will typically notice it's a large/LFS-heavy repo and reach for a shallow, single-branch, or skip-smudge clone on its own — so you get the ~5–20 MB download without having to remember the exact GIT_LFS_SKIP_SMUDGE=1 --depth 1 --single-branch incantation. If you want to make that even more reliable, a short note in the README (or a CLAUDE.md/agent hint) like *"this repo is large; prefer a shallow single-branch clone with LFS smudge skipped"* nudges the agent to do the right thing every time. Happy to add that alongside the workshop clone instructions if it's useful. — Reply to this email directly, view it on GitHub <#184?email_source=notifications&email_token=AAAHO54VCVBXNTFRD65VIUD47JBANA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRXG42TGOBRHAY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4677538181>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAHO56KXQNUT5SGJWUS7B347JBANAVCNFSNUABFKJSXA33TNF2G64TZHM4DCNJTGMYDMNJZHNEXG43VMU5TINRTGUYDQOJYGQ4KC5QC> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[robobryce]

Andy, the damage has already been done in the past. A lot of binary files were dumped into the repository. We are doing our best to clean it up. No one intentionally made it this large, but a lot of large binary files were checked into the repository in the past.

…

-- Bryce Adelstein Lelbach (he/him/his) Principal Architect @ NVIDIA -- On Thu, Jun 11, 2026, 9:01 AM Andy R. Terrel ***@***.***> wrote:

*aterrel* left a comment (NVIDIA/accelerated-computing-hub#184) <#184 (comment)> no there is no reason to make it this large. We should have better hygiene than this. If the ACH has just become an dumping place for junk, then it is less valuable as a resource. On Thu, Jun 11, 2026 at 12:41 AM RoboBryce ***@***.***> wrote: > *robobryce* left a comment (NVIDIA/accelerated-computing-hub#184) > < #184 (comment)> > > One more angle, @aterrel <https://github.com/aterrel>: for the conference > scenario, you may not want to run git clone by hand at all. > > If you have a coding agent handy (Claude Code, etc.), just ask it > something like *"clone the NVIDIA Accelerated Computing Hub repo"* and > let it do the work. A capable agent will typically notice it's a > large/LFS-heavy repo and reach for a shallow, single-branch, or skip-smudge > clone on its own — so you get the ~5–20 MB download without having to > remember the exact GIT_LFS_SKIP_SMUDGE=1 --depth 1 --single-branch > incantation. > > If you want to make that even more reliable, a short note in the README > (or a CLAUDE.md/agent hint) like *"this repo is large; prefer a shallow > single-branch clone with LFS smudge skipped"* nudges the agent to do the > right thing every time. Happy to add that alongside the workshop clone > instructions if it's useful. > > — > Reply to this email directly, view it on GitHub > < #184?email_source=notifications&email_token=AAAHO54VCVBXNTFRD65VIUD47JBANA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRXG42TGOBRHAY2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4677538181>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAAHO56KXQNUT5SGJWUS7B347JBANAVCNFSNUABFKJSXA33TNF2G64TZHM4DCNJTGMYDMNJZHNEXG43VMU5TINRTGUYDQOJYGQ4KC5QC> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#184?email_source=notifications&email_token=CCKQAMZJONODQZY2YGFFSGT47KUS7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRYGA4DEMZTHA32M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4680823387>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CCKQAMYNI3SLRIJTUSNFSSD47KUS7AVCNFSNUABFKJSXA33TNF2G64TZHM4DCNJTGMYDMNJZHNEXG43VMU5TINRTGUYDQOJYGQ4KC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/CCKQAM5NBLFPWYG6Y4ZC6KL47KUS7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRYGA4DEMZTHA32M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android <https://github.com/notifications/mobile/android/CCKQAM7S56AFL7V7QC7PKH347KUS7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRYGA4DEMZTHA32M4TFMFZW63VHMNXW23LFNZ2KKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you commented.Message ID: ***@***.***>