Skip to content

files: batch link creation and target checks #9718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrewgazelka wants to merge 1 commit into
base: master
Choose a base branch
from
+124 −24
Open

files: batch link creation and target checks #9718

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 74 additions & 1 deletion modules/files.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,80 @@ in

newGenFiles="$1"
shift

# Classify every target using bash builtins only: even one forked
# process per file costs several milliseconds on some platforms
# (notably darwin), which dominates activation time when a profile
# carries hundreds of links. Targets occupied by a regular file or
# directory keep the original per-file handling (backup and
# identical-content skip) on the slow path below.
declare -a symlinkTargets=() symlinkSources=()
declare -a linkSources=() linkDirs=()
declare -a slowSources=()
for sourcePath in "$@" ; do
relativePath="''${sourcePath#$newGenFiles/}"
targetPath="$HOME/$relativePath"
if [[ -L "''${targetPath%/*}" ]] ; then
# The parent directory is itself a symlink (e.g. a stale
# whole-directory link from an older layout). The batched
# `ln -n -t` below would refuse it ("Not a directory"), while
# the original per-file `ln -T` traverses it; keep upstream
# behavior on the slow path.
slowSources+=("$sourcePath")
elif [[ -L "$targetPath" ]] ; then
symlinkTargets+=("$targetPath")
symlinkSources+=("$sourcePath")
elif [[ -e "$targetPath" ]] ; then
slowSources+=("$sourcePath")
else
linkSources+=("$sourcePath")
linkDirs+=("''${targetPath%/*}")
fi
done

# Resolve all existing symlinks with a single readlink call and
# relink only those not already pointing at the new generation.
if [[ ''${#symlinkTargets[@]} -gt 0 ]] ; then
i=0
while IFS= read -r -d "" currentSource ; do
if [[ "$currentSource" != "''${symlinkSources[i]}" ]] ; then
linkSources+=("''${symlinkSources[i]}")
linkDirs+=("''${symlinkTargets[i]%/*}")
fi
i=$(( i + 1 ))
done < <(readlink -z -- "''${symlinkTargets[@]}")
Comment on lines +229 to +235

@khaneliman khaneliman Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (blocking): reject partial readlink batches before linking

readlink omits a record when an operand vanishes, so a later result shifts onto the wrong source; I reproduced the first target disappearing, the helper exiting 0, and the second link staying stale. Minimum safe fix is an exact-count guard immediately after this loop, before the later mkdir/ln calls:

if [[ $i -ne ''${#symlinkTargets[@]} ]] ; then
  errorEcho "A link target changed while resolving symlinks; retry activation."
  exit 1
fi

Please mirror the equivalent guard in check-link-targets.sh; reclassifying or falling back per target instead of exiting is also valid.

fi

# Create all missing parent directories in one mkdir call.
declare -A missingDirs=()
for targetDir in "''${linkDirs[@]}" ; do
[[ -d "$targetDir" ]] || missingDirs[$targetDir]=1
done
if [[ ''${#missingDirs[@]} -gt 0 ]] ; then
run mkdir -p $VERBOSE_ARG -- "''${!missingDirs[@]}" || exit 1
fi

# Group the pending links by parent directory, one ln call per
# directory. The link name always equals the source basename, and
# -f -n together replace a stale symlink even when it points at a
# directory (the case -T guarded against in the per-file version;
# a regular directory in the way takes the slow path instead and
# fails there just like it always did).
declare -A dirBatches=()
for i in "''${!linkSources[@]}" ; do
dirBatches[''${linkDirs[i]}]+="$i "
done
for targetDir in "''${!dirBatches[@]}" ; do
batch=()
for i in ''${dirBatches[$targetDir]} ; do
batch+=("''${linkSources[i]}")
done
run ln -sfn $VERBOSE_ARG -t "$targetDir" -- "''${batch[@]}" || exit 1
done

# Slow path: the target exists and is not a symlink. This is the
# original per-file logic, kept verbatim for the rare collisions.
for sourcePath in "''${slowSources[@]}" ; do
relativePath="''${sourcePath#$newGenFiles/}"
targetPath="$HOME/$relativePath"
if [[ -e "$targetPath" && ! -L "$targetPath" ]] ; then
Expand All @@ -209,7 +282,7 @@ in
fi

if [[ -e "$targetPath" && ! -L "$targetPath" ]] && cmp -s "$sourcePath" "$targetPath" ; then
# The target exists but is identical – don't do anything.
# The target exists but is identical - don't do anything.
verboseEcho "Skipping '$targetPath' as it is identical to '$sourcePath'"
else
# Place that symlink, --force
Expand Down
73 changes: 50 additions & 23 deletions modules/files/check-link-targets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,39 @@ forcedPaths=(@forcedPaths@)

newGenFiles="$1"
shift

# Check a target that already exists and is not a symlink owned by Home
# Manager.
function checkCollision() {
local sourcePath="$1"
local targetPath="$2"

if cmp -s "$sourcePath" "$targetPath"; then
# First compare the files' content. If they're equal, we're fine.
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath', will be skipped since they are the same"
elif [[ ! -L "$targetPath" && -n "$HOME_MANAGER_BACKUP_COMMAND" ]] ; then
# Next, try to run the custom backup command. Assume this always succeeds.
verboseEcho "Existing file '$targetPath' exists and differs from '$sourcePath'. '$HOME_MANAGER_BACKUP_COMMAND' will be used to backup the file."
elif [[ ! -L "$targetPath" && -n "$HOME_MANAGER_BACKUP_EXT" ]] ; then
# Next, try to move the file to a backup location if configured and possible
backup="$targetPath.$HOME_MANAGER_BACKUP_EXT"
if [[ -e "$backup" && -z "$HOME_MANAGER_BACKUP_OVERWRITE" ]] ; then
collisionErrors+=("Existing file '$backup' would be clobbered by backing up '$targetPath'")
elif [[ -e "$backup" && -n "$HOME_MANAGER_BACKUP_OVERWRITE" ]] ; then
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath' and '$backup' exists. Backup will be clobbered due to HOME_MANAGER_BACKUP_OVERWRITE=1"
else
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath', will be moved to '$backup'"
fi
else
# Fail if nothing else works
collisionErrors+=("Existing file '$targetPath' would be clobbered")
fi
}

# Classify every target using bash builtins only: even one forked process
# per file costs several milliseconds on some platforms (notably darwin),
# which dominates activation time when a profile carries hundreds of links.
declare -a linkTargets=() linkSources=()
for sourcePath in "$@" ; do
relativePath="${sourcePath#$newGenFiles/}"
targetPath="$HOME/$relativePath"
Expand All @@ -24,32 +57,26 @@ for sourcePath in "$@" ; do

if [[ -n $forced ]]; then
verboseEcho "Skipping collision check for $targetPath"
elif [[ -e "$targetPath" \
&& ! "$(readlink "$targetPath")" == $homeFilePattern ]] ; then
# The target file already exists and it isn't a symlink owned by Home Manager.
if cmp -s "$sourcePath" "$targetPath"; then
# First compare the files' content. If they're equal, we're fine.
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath', will be skipped since they are the same"
elif [[ ! -L "$targetPath" && -n "$HOME_MANAGER_BACKUP_COMMAND" ]] ; then
# Next, try to run the custom backup command. Assume this always succeeds.
verboseEcho "Existing file '$targetPath' exists and differs from '$sourcePath'. '$HOME_MANAGER_BACKUP_COMMAND' will be used to backup the file."
elif [[ ! -L "$targetPath" && -n "$HOME_MANAGER_BACKUP_EXT" ]] ; then
# Next, try to move the file to a backup location if configured and possible
backup="$targetPath.$HOME_MANAGER_BACKUP_EXT"
if [[ -e "$backup" && -z "$HOME_MANAGER_BACKUP_OVERWRITE" ]] ; then
collisionErrors+=("Existing file '$backup' would be clobbered by backing up '$targetPath'")
elif [[ -e "$backup" && -n "$HOME_MANAGER_BACKUP_OVERWRITE" ]] ; then
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath' and '$backup' exists. Backup will be clobbered due to HOME_MANAGER_BACKUP_OVERWRITE=1"
else
warnEcho "Existing file '$targetPath' is in the way of '$sourcePath', will be moved to '$backup'"
fi
else
# Fail if nothing else works
collisionErrors+=("Existing file '$targetPath' would be clobbered")
fi
elif [[ -L "$targetPath" && -e "$targetPath" ]] ; then
linkTargets+=("$targetPath")
linkSources+=("$sourcePath")
elif [[ -e "$targetPath" ]] ; then
checkCollision "$sourcePath" "$targetPath"
fi
done

# Resolve all existing symlinks with a single readlink call. A link into a
# Home Manager generation is ours; anything else is a collision candidate.
if [[ ${#linkTargets[@]} -gt 0 ]] ; then
i=0
while IFS= read -r -d "" currentSource ; do
if [[ ! "$currentSource" == $homeFilePattern ]] ; then
checkCollision "${linkSources[i]}" "${linkTargets[i]}"
fi
i=$(( i + 1 ))
done < <(readlink -z -- "${linkTargets[@]}")
fi

if [[ ${#collisionErrors[@]} -gt 0 ]] ; then
errorEcho "Please do one of the following:
- In standalone mode, use 'home-manager switch -b backup' to back up"\
Expand Down
Loading