diff --git a/CONTRIBUTION.md b/CONTRIBUTION.md new file mode 100644 index 000000000..2c8e0d381 --- /dev/null +++ b/CONTRIBUTION.md @@ -0,0 +1,302 @@ +# Contribution 1: Live Preview / Hot-Reloading for Copier Templates + +**Contribution Number:** 1 +**Student:** Karen Emily Muhwezi +**Issue:** https://github.com/copier-org/copier/issues/1451 +**Status:** Phase IV Pending + +--- + +## Why I Chose This Issue + +As someone with a Python background who has worked with ML models, I'm drawn +to tooling problems that slow developers down in real, tangible ways. This +issue describes a genuinely frustrating workflow, having to commit, push, and +run updates just to preview a template change, and proposes a solution that +would make the development experience significantly better. That kind of +quality-of-life improvement is something I find meaningful to work on because +the impact is immediately visible. + +This issue also aligns well with my learning goals. Copier is a Python +project, which plays to my existing strengths, but building a live preview +feature means I'll be working with file watching, CLI design, and real-time +rendering pipelines, areas I haven't worked in deeply before. I want to grow +my problem-solving skills by tackling something that has genuine complexity +underneath a simple-sounding description, and see that solution actually ship +in a tool that real developers use every day. + +--- + +## Understanding the Issue + +### Problem Description + +Copier currently has no way to preview how a template renders during +development. To see any changes, a developer must commit their changes, push +to GitHub, switch to a downstream project, run `copier update`, and observe +the output, then repeat the entire cycle for every fix. This makes template +development extremely slow and tedious. + +### Expected Behavior + +A template developer should be able to run a single command like +`copier preview` that renders their template locally into a temporary +directory and automatically re-renders it every time they save a change, +without any manual steps in between, similar to hot-reloading in React +development. + +### Current Behavior + +There is no preview or watch command in Copier. The only way to test template +changes is to go through the full commit, push, and `copier update` cycle on +a downstream project, which requires switching between multiple repositories +and running multiple commands every single time a change is made. + +### Affected Components + +- Copier's CLI layer — a new `preview` or `watch` command needs to be added +- Core template rendering pipeline in `copier/_main.py` — the existing render + logic will be reused +- A new file watching component using the `watchfiles` library to detect + changes and trigger re-renders + +--- + +## Reproduction Process + +### Environment Setup + +Cloned fork from https://github.com/karenemily/codepath-copier.git + +Ran `python -m uv sync` and hit a network timeout trying to fetch `hatch_vcs`: + +x Failed to build copier @ file:///C:/Users/Administrator/codepath-copier +|-> Failed to resolve requirements from build-system.requires +|-> No solution found when resolving: hatchling, hatch-vcs +|-> Request failed after 3 retries in 84.0s +|-> error sending request for url (https://files.pythonhosted.org/...) +`-> operation timed out + +Resolved by retrying on a stable network connection. Confirmed working +environment by running `python -m uv run copier --version` which returned +`copier 0.1.dev2247+g735b2ed0e`. + +Note: `uv` is not on the system PATH on Windows, so all commands must be +run as `python -m uv` instead of `uv` directly. + +### Steps to Reproduce + +1. Create a Copier template locally +2. Make a change to the template +3. Attempt to preview how the change renders without pushing to GitHub +4. Observed result: No preview mechanism exists — must commit, push, and run + `copier update` on a downstream project to see any changes + +### Reproduction Evidence + +- **Commit showing reproduction:** https://github.com/karenemily/codepath-copier/commit/65b7fcaad88c0ac22ffe5a245e03d74a056789f8 +- **Screenshots/logs:** Network timeout error documented above +- **My findings:** The issue is a missing feature — there is no existing + preview or watch command in Copier's CLI. Confirmed by searching the + codebase for any `preview` or `watch` functionality and finding none. + Identified `_main.py` as the correct file to add the feature by studying + how `run_copy` and `run_update` are structured. Confirmed the feature + was fully missing by successfully running `copier --help` and seeing no + preview command listed. + +--- + +## Solution Approach + +### Analysis + +This is a missing feature rather than a bug. Copier currently only supports +rendering templates on demand via `copier copy` or `copier update`. There is +no mechanism to watch a template directory for changes and re-render +automatically. The root cause is that the CLI and rendering pipeline were +built for one-shot use, not continuous development workflows. + +### Proposed Solution + +Added a new `run_preview` function that renders the template into a +destination directory and then watches the template directory for file +changes using the `watchfiles` library, re-rendering automatically on every +change detected. + +### Implementation Plan + +Using UMPIRE framework (adapted): + +**Understand:** Template developers currently have no way to preview renders +locally without a full commit-push-update cycle. We need a live preview +command that watches for changes and re-renders automatically. + +**Match:** The existing `run_copy` and `run_update` methods in `copier/_main.py` +show how rendering is triggered inside the `Worker` class. The new +`run_preview` method follows the exact same pattern, reusing `_render_template()` +and wrapping it in a `watchfiles` loop. The standalone function pattern was +modelled on `run_recopy`. + +**Plan:** +1. ✅ Add `watchfiles` as a dependency in `pyproject.toml` +2. ✅ Add `from watchfiles import watch` to module-level imports in `_main.py` +3. ✅ Add `run_preview` method to the `Worker` class decorated with + `@as_operation("copy")` +4. ✅ Add standalone `run_preview` function following the same pattern as + `run_recopy` +5. ✅ Write and pass test for the new command +6. ✅ Expose the new function as a CLI command via the existing CLI layer +7. ⬜ Add additional edge case tests + +**Implement:** https://github.com/karenemily/codepath-copier/tree/fix-issue-live-preview + +**Review:** +- [x] Follows Copier's code style +- [x] New command is documented in PR description +- [x] Existing tests still pass +- [x] New tests added and passing +- [ ] CHANGELOG updated if required by maintainer + +**Evaluate:** Test `test_preview_renders_template` confirms the initial render +works correctly. Manual end-to-end testing confirmed hot-reload re-renders +correctly when template files are saved. + +--- + +## Testing Strategy + +### Unit Tests + +- [x] Test case 1: `test_preview_renders_template` — confirms that + `run_preview` correctly renders template files into the destination + directory on initial run. Test passes. +- [ ] Test case 2: Re-render triggered correctly after a file change is + detected by the watcher +- [ ] Test case 3: Preview handles template errors gracefully without + crashing the watcher + +### Integration Tests + +- [ ] Integration scenario 1: Full preview workflow from template edit to + re-render +- [ ] Integration scenario 2: Preview works correctly with Jinja templating + and variable substitution + +### Manual Testing + +Manually tested by running `copier preview` on a local test template. Created +a template with a `hello.txt` file, ran the preview command, edited the file, +and confirmed the output directory updated automatically without any manual +steps. Hot-reload confirmed working end to end. + +--- + +## Implementation Notes + +### Week 1 Progress + +- Reviewed `CONTRIBUTING.md` and understood code style, testing, and commit + message requirements (Conventional Commits format) +- Explored codebase — identified `_main.py` as the correct file, studied + `run_copy` and `run_update` patterns +- Added `watchfiles>=0.20` to `pyproject.toml` dependencies +- Added `from watchfiles import watch` to module-level imports in `_main.py` +- Added `run_preview` method to the `Worker` class using `@as_operation("copy")` + decorator to satisfy operation context requirements +- Added standalone `run_preview` function with `src_path`, `dst_path`, `data`, + `defaults`, `overwrite`, and `quiet` parameters +- Wrote and passed test `test_preview_renders_template` in `tests/test_copy.py` +- Debugged multiple errors including SyntaxError from misplaced import, + LookupError from missing operation context, and mock patching issues + +### Week 2 Progress + +- Added `CopierPreviewSubApp` class to `copier/_cli.py` to expose `run_preview` + as a `copier preview` CLI subcommand +- Imported `run_preview` into the CLI module alongside existing commands +- Rebased branch on `upstream/master` to stay up to date +- Ran full test suite — 246 passed, 2 pre-existing Windows-only failures + unrelated to our changes +- Performed manual end-to-end testing — confirmed hot-reload works correctly +- Submitted PR #2747 to the upstream Copier repository +- Tagged maintainer @sisp for review + +### Code Changes + +- **Files modified:** `copier/_main.py`, `copier/_cli.py`, `pyproject.toml`, + `tests/test_copy.py` +- **Key commits:** https://github.com/karenemily/codepath-copier/tree/fix-issue-live-preview +- **Approach decisions:** Used `@as_operation("copy")` decorator on the + `run_preview` method to satisfy the internal operation context requirement. + Mocked `watchfiles.watch` in tests to avoid actual file watching during + test runs. Moved `watchfiles` import to module level to allow proper mocking. + Followed the existing `CopierCopySubApp` pattern exactly when building the + CLI subcommand. + +--- + +## Pull Request + +**PR Link:** https://github.com/copier-org/copier/pull/2747 + +**PR Description:** Added a new `copier preview` CLI command that renders +a template locally and automatically re-renders it whenever a file change +is detected using `watchfiles`, implementing the hot-reloading feature +requested in issue #1451. + +**Maintainer Feedback:** +"I don't think the current implementation is a correct solution to the problem. + +It is insufficient to run Worker._render_template on source file changes – this ignores changes in the questionnaire, which likely affects the render context, and doesn't (re)run tasks, which may be essential to the render output. +It doesn't remove files that were previously rendered but are no longer rendered after a template change – files are only added or replaced. I imagine a (more) correct solution might involve making a full copy (using run_copy) in a temporary destination, computing the filetree diff between the temporary copy and the user-specified destination, and syncing the latter to the former. But this is just the first idea that has come to my mind right now. I imagine the devil is in the details. For example, for good UX you'll likely want to ask only questions that were changed in copier.yml, but a template might not render an answers file which is what copier update --skip-answered uses. On [CONTRIBUTION.md](https://github.com/copier-org/copier/pull/2747#discussion_r3529591928): I don't understand this file, it doesn't belong in this repo." + +**Status:** Awaiting review and working on implementing the feedback + +--- + +## Learnings & Reflections + +### Technical Skills Gained + +- Learned how to navigate a large unfamiliar Python codebase +- Understood the `Worker` class pattern and how Copier's operations are + structured +- Learned how to use `unittest.mock` to mock third-party library calls in tests +- Learned about Python `ContextVar` and operation context requirements +- Practised Conventional Commits format for commit messages +- Learned how CLI frameworks work and how to add new subcommands +- Gained experience with Git rebasing and working with upstream remotes +- Learned how to submit a real open source pull request + +### Challenges Overcome + +- `uv` not on PATH on Windows — resolved by using `python -m uv` prefix +- Network timeout during `uv sync` — resolved by retrying on stable network +- SyntaxError from placing `from watchfiles import watch` before + `from __future__ import annotations` — resolved by moving it to the correct + position in the imports section +- `LookupError` for missing operation context — resolved by adding + `@as_operation("copy")` decorator to the method +- Mock patching failing because import was local — resolved by moving import + to module level +- Git rebase failing because default branch is `master` not `main` — resolved + by using `upstream/master` + +### What I'd Do Differently Next Time + +- Set up the development environment on a Linux or Mac machine to avoid + Windows-specific issues with symlinks, PATH, and missing commands like `cat` +- Comment on the issue before starting work to confirm it is still wanted + by maintainers and get implementation guidance early +- Write tests alongside the code rather than after to catch issues sooner + +--- + +## Resources Used + +- https://github.com/copier-org/copier — Main repository +- https://github.com/copier-org/copier/issues/1451 — Issue being addressed +- https://github.com/copier-org/copier/pull/2747 — Submitted pull request +- https://github.com/samuelcolvin/watchfiles — Suggested file watching library +- Copier documentation: https://copier.readthedocs.io +- Copier CONTRIBUTING.md — Code style, testing and commit message guidelines diff --git a/copier/_cli.py b/copier/_cli.py index 0365f9364..0c2ff93c3 100644 --- a/copier/_cli.py +++ b/copier/_cli.py @@ -67,7 +67,7 @@ import yaml from plumbum import LocalPath, cli, colors -from ._main import get_update_data, run_copy, run_recopy, run_update +from ._main import get_update_data, run_copy, run_preview, run_recopy, run_update from ._tools import copier_version, try_enum from ._types import AnyByStrDict, VcsRef from .errors import UnsafeTemplateError, UserMessageError @@ -565,3 +565,46 @@ def inner() -> int: return 0 return _handle_exceptions(inner) + +@CopierApp.subcommand("preview") +class CopierPreviewSubApp(_Subcommand): + """The `copier preview` subcommand. + + Use this subcommand to preview how your template renders as you develop + it, with hot-reloading whenever you save a change. + """ + + DESCRIPTION = "Preview a template with hot-reloading as you develop it." + + defaults = cli.Flag( + ["-l", "--defaults"], + help="Use default answers to questions, which might be null if not specified.", + ) + overwrite = cli.Flag( + ["-w", "--overwrite"], + default=True, + help="Overwrite files that already exist.", + ) + + def main(self, template_src: str, destination_path: str) -> int: + """Call [run_preview][copier.run_preview]. + + Params: + template_src: + Path to the template you are developing locally. + + destination_path: + Where to render the preview output. + """ + + def inner() -> None: + run_preview( + template_src, + destination_path, + data=self.data, + defaults=self.defaults, + overwrite=self.overwrite, + quiet=self.quiet, + ) + + return _handle_exceptions(inner) diff --git a/copier/_main.py b/copier/_main.py index a584dba48..e1bdcfb05 100644 --- a/copier/_main.py +++ b/copier/_main.py @@ -1,7 +1,9 @@ """Main functions and classes, used to generate or update projects.""" + from __future__ import annotations + import os import platform import stat @@ -18,6 +20,7 @@ from itertools import chain from pathlib import Path, PurePath, PurePosixPath, PureWindowsPath from shutil import rmtree +from watchfiles import watch from tempfile import TemporaryDirectory from types import TracebackType from typing import ( @@ -1358,6 +1361,29 @@ def run_update(self) -> None: self._apply_update() self._print_message(self.template.message_after_update) + + @as_operation("copy") + def run_preview(self, dst_path: Path) -> None: + """Preview a template with hot-reloading as changes are made.""" + + print(f"\nStarting preview of template at {self.template.local_abspath}") + print("Watching for changes... (Ctrl+C to stop)\n") + + # Do initial render + with Phase.use(Phase.RENDER): + self._render_template() + print("Initial render complete.") + + # Watch for changes and re-render + try: + for changes in watch(self.template.local_abspath): + print(f"\nDetected changes: {changes}") + print("Re-rendering template...") + with Phase.use(Phase.RENDER): + self._render_template() + print("Re-render complete.") + except KeyboardInterrupt: + print("\nPreview stopped.") def _apply_update(self) -> None: # noqa: C901 git = get_git() @@ -1805,6 +1831,26 @@ def run_recopy( worker.run_recopy() return worker +def run_preview( + src_path: Path | str, + dst_path: Path | str = ".", + data: dict[str, Any] | None = None, + defaults: bool = True, + overwrite: bool = True, + quiet: bool = False, +) -> Worker: + """Preview a template with hot-reloading as you develop it.""" + with Worker( + src_path=src_path, + dst_path=Path(dst_path), + data=data or {}, + defaults=defaults, + overwrite=overwrite, + quiet=quiet, + ) as worker: + worker.run_preview(Path(dst_path)) + return worker + def run_update( dst_path: Path | str = ".", @@ -1864,6 +1910,7 @@ def run_update( return worker + def get_update_data( dst_path: Path | str = ".", answers_file: Path | str | None = None, diff --git a/pyproject.toml b/pyproject.toml index a037a8bb7..aa4d4e008 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -32,6 +32,7 @@ dependencies = [ "questionary>=1.8.1", "platformdirs>=4.3.6", "typing-extensions>=4.0.0,<5.0.0; python_version < '3.11'", + "watchfiles>=0.20", ] [project.urls] diff --git a/tests/test_copy.py b/tests/test_copy.py index 24e4c2b6a..5b90db15f 100644 --- a/tests/test_copy.py +++ b/tests/test_copy.py @@ -1522,3 +1522,21 @@ def test_copy_with_relative_path_to_local_git_template_through_multiple_parents( assert (dst / "test.txt").is_file() assert (dst / "test.txt").read_text() == "test" + +def test_preview_renders_template(tmp_path_factory: pytest.TempPathFactory) -> None: + """Test that run_preview renders the template into the destination directory.""" + src, dst = map(tmp_path_factory.mktemp, ("src", "dst")) + build_file_tree( + { + src / "hello.txt": "Hello, world!", + src / "copier.yml": "", + } + ) + # We mock watch to avoid actually watching for file changes in tests + with mock.patch("copier._main.watch") as mock_watch: + mock_watch.return_value = [] + from copier._main import run_preview + run_preview(str(src), dst, defaults=True, overwrite=True) + + assert (dst / "hello.txt").exists() + assert (dst / "hello.txt").read_text() == "Hello, world!" \ No newline at end of file