Sync addons-source@maintenance/gramps60 with upstream (2026-07-25) - #62
Sync addons-source@maintenance/gramps60 with upstream (2026-07-25)#62eduralph wants to merge 51 commits into
Conversation
Seventeen manual pages for addon authors - overview and getting started, tutorials per addon kind, the addon-kinds catalogue, registration fundamentals, data access, API reference, testing, debugging, troubleshooting, code analysis, internationalization, packaging, post-merge community steps, compatibility, per-release changes, normative guidelines, and roadmap - plus the diagrams they embed and a folder index. The new docs/ tree is invisible to make.py and CI: every enumeration keys on *.gpr.py or *.py globs that match nothing under docs/, verified with manifest-check and a no-op 'build docs' run.
README: the develop-your-own-addon pointer now leads to the in-repo docs/addon-development manual first, with CONTRIBUTING.md for the contributor workflow and the wiki page as an alternative rendering. The dead Travis badge is dropped. CONTRIBUTING: the deep technical sections that the manual now covers - addon kinds, registration and GENERAL plugins, prerequisites, addon configuration, localization, distribution contents, report categories, and the wiki listing/documentation templates - are reduced to a short retained summary plus a link into the manual, each under its original heading so existing deep links keep resolving. The contributor-workflow content that is unique to this document - repository and fork setup, development branches, the addon checklist, the pull-request walkthrough, and the maintenance guidance - is kept in place unchanged. Also repairs pre-existing broken links: seven table-of-contents and overview anchors that never matched their headings, two (#https://...) hrefs, the garbled Addon-list-legend link, and the Localization snippet that had lost its underscore binding. Depends on the PR that adds docs/addon-development.
The addons-source CI (PR 820) adds a lint step that fails on any tracked Python file carrying trailing whitespace. Three pre-existing files trip it (27 lines total): ArchiveAssist/ArchiveAssist.py (22), and two FilterRules modules (5). Strip the trailing whitespace so the gate can pass; whitespace only, no behavioural change (git diff -w is empty).
_build_with_progress used `for _ in range(batch_size)`, which makes _
a local variable throughout the whole method (Python binds it at compile
time). That shadows the module-level `_ = _trans.gettext`, so:
- the except handler at the top of the method, LOG.error(_("Error
initializing data: %s") % str(e)), runs before the loop assigns _
and raises UnboundLocalError; and
- the in-loop handler LOG.warning(_("Error processing person %s: %s"
% ...)) calls _ after the loop bound it to an int, raising TypeError.
Both fire only on the error paths, so they slipped through. The loop
counter is unused; rename it to _batch_step so _ resolves to the module
gettext everywhere. ruff (E9,F63,F7,F82) is clean on the module.
This also clears the F82x lint error PR 820's ruff gate reports on the
current tree.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f1011fa25
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| register( | ||
| QUICKVIEW, | ||
| id="Siblings", |
There was a problem hiding this comment.
Register Quick Views with QUICKREPORT
On Gramps 6.0, the injected registration constant is QUICKREPORT, not QUICKVIEW; every working Quick View in this branch, such as AllNamesQuickview/AllNames.gpr.py, uses QUICKREPORT. Following this tutorial therefore raises NameError while loading the .gpr.py, so the example never appears in the UI.
Useful? React with 👍 / 👎.
|
|
||
| 2. **`gramps_target_version` mismatch.** A `6.0` addon won't load in 6.1, and vice versa. Plugin discovery silently skips the registration entry. See [14-compatibility → `gramps_target_version` semantics](14-compatibility.md#gramps_target_version-semantics). | ||
|
|
||
| 3. **`id` doesn't match the folder name.** The addon's folder name and the `id` argument to `register(...)` must be identical. Gramps does not match by content — it matches by folder name and verifies against `id`. A mismatch silently drops the entry. |
There was a problem hiding this comment.
Remove the folder-name requirement for plugin ids
Plugin IDs need to be unique but do not have to equal the containing folder name, as this manual correctly states elsewhere and existing addons demonstrate (for example, folder AllNamesQuickview registers id="allnames"). Treating a mismatch as a loading failure sends users toward an unnecessary rename while overlooking the real registration error.
Useful? React with 👍 / 👎.
|
|
||
| 3. **`id` doesn't match the folder name.** The addon's folder name and the `id` argument to `register(...)` must be identical. Gramps does not match by content — it matches by folder name and verifies against `id`. A mismatch silently drops the entry. | ||
|
|
||
| The fastest check: in a Python REPL with `gramps` on `sys.path`, `exec(open("MyAddon/MyAddon.gpr.py").read())`. If it raises, you have your cause; if it returns silently and there's no entry, your `register()` call is being filtered out. |
There was a problem hiding this comment.
Execute registration files with the loader environment
A plain exec in a REPL does not provide the names that Gramps injects into .gpr.py, including register, _, GRAMPLET, and the status constants. Consequently this suggested check raises NameError for every valid registration file and falsely reports a registration defect; the diagnostic must supply make_environment/a stub register, or use the existing registration checker.
Useful? React with 👍 / 👎.
|
|
||
| # 1. Iterate: edit, restart Gramps, verify behaviour. | ||
| # 2. Refresh translations if user-visible strings changed. | ||
| make.py gramps60 update <Addon> all |
There was a problem hiding this comment.
Pass a real locale to the update command
make.py does not support all for the update command: it assigns the fifth argument directly to locale and requires <Addon>/po/<locale>-local.po, so this invocation fails looking for po/all-local.po. Authors following the update workflow cannot refresh translations unless the instructions specify each locale separately or add actual all handling.
Useful? React with 👍 / 👎.
|
|
||
| ## Translating UI Files (Glade) | ||
|
|
||
| Gramps' addon translation tools only automatically extract and manage Python strings. If your addon uses a Glade (`.ui` / `.glade`) file for its interface, those strings will not be picked up by the standard addon translation workflow. The recommended pattern is to mark the Glade strings as translatable (so they show up for translators), then override the label at runtime from Python so they get translated through your addon's gettext domain. |
There was a problem hiding this comment.
Document the existing Glade extraction step
For top-level .glade files this statement is incorrect: make.py lines 450–455 glob *.glade and append their translatable strings to template.pot with xgettext -L Glade. Telling authors that the standard workflow ignores those strings encourages redundant runtime label overrides and misrepresents what make.py init produces; only unsupported UI-file cases should be called out.
Useful? React with 👍 / 👎.
| Local invocation: | ||
|
|
||
| ```bash | ||
| ruff check --select E9,F63,F7,F82 <Addon>/ |
There was a problem hiding this comment.
Exclude registration files from the local Ruff command
Checked the repository's CI command at .github/workflows/ci.yml: it explicitly passes --exclude='*.gpr.py' because injected names in registration files are intentionally undefined to static analysis. The documented command omits that exclusion, so running it on a valid addon reports F821 for register, _, plugin-kind constants, and status constants rather than mirroring the CI gate.
Useful? React with 👍 / 👎.
| - **MUST**: `fname` points to an implementation module shipped in the same folder. | ||
| - **MUST**: the addon is physically present under the plugin path — a physical copy works on every Gramps version and OS. (Gramps 6.1+ also discovers an addon reached via a symlink, but a physical copy is the portable default.) | ||
| - **MUST NOT**: import `register`, `GRAMPLET`, `STABLE`, `_`, or any other name Gramps injects into the `.gpr.py` namespace. | ||
| - **MUST NOT**: add `__init__.py` to the addon directory itself. The plugin loader puts the addon dir on `sys.path` and imports `<Addon>.py` by name; making the addon dir a regular package disturbs that resolution and can trigger the [Mantis 12691](https://gramps-project.org/bugs/view.php?id=12691) submodule-binding trap. (See [07-testing → Why `tests/__init__.py` exists](07-testing.md#why-tests__init__py-exists).) |
There was a problem hiding this comment.
Allow package-based addon directories
This repository already contains functioning addon roots with __init__.py, including Sqlite, Query, DynamicWeb, PlaceCleanup, and PostgreSQLEnhanced; the test runner also explicitly handles regular addon packages. Making their absence an unconditional MUST would tell maintainers to remove files needed for package-relative imports and tests, so the prohibition should be limited to the specific same-name module collision it is meant to prevent.
Useful? React with 👍 / 👎.
|
|
||
| - **MUST**: wrap every user-visible string with `_()`. | ||
| - **MUST NOT**: `import _` in `.gpr.py` — Gramps' plugin loader injects it. Implementation modules **MUST** bind it explicitly via `_ = glocale.get_addon_translator(__file__).gettext`. | ||
| - **MUST** (multi-file packages): when the addon's code is split across a nested package, bind `_` **once at the addon root** — the directory that holds `locale/`, in a root-level module (e.g. `_i18n.py`) — and import it everywhere else by **bare name** (`from _i18n import _`), **not** a `<Addon>.`-prefixed path: the addon dir is on `sys.path` and its root is **not** a package (see *Structure* → MUST NOT `__init__.py`), so a root-level module imports directly, whereas `from <Addon>.<pkg>.i18n import _` raises `'<Addon>' is not a package` at import time. `get_addon_translator(filename)` derives the catalog dir as `dirname(abspath(filename)) + "/locale"` (`gramps.gen.utils.grampslocale`), so a `get_addon_translator(__file__)` call from a nested module (e.g. `myaddon/views/tab.py`) resolves `myaddon/views/locale/`, which doesn't exist, and a non-English user silently gets the untranslated string. The flat `_ = glocale.get_addon_translator(__file__).gettext` form above is correct only because that module sits at the addon root; from a nested module, anchor the path at the root (e.g. `get_addon_translator(os.path.join(ADDON_ROOT, "_"))` — only `dirname(...)` is read, so the basename is an unused placeholder) instead of passing `__file__`. (NameSuite i18n-anchor fix, 2026-06-25.) |
There was a problem hiding this comment.
Give the shared translation module an addon-unique name
If two loaded addons follow this example and both create a top-level _i18n.py, Python caches the first one as sys.modules['_i18n']; the second addon's bare from _i18n import _ then reuses the first addon's translator and catalog. Plugin-path removal does not clear sys.modules, so the recommended generic module name creates cross-addon translation contamination; use an addon-unique top-level name or a genuinely namespaced package such as the existing NameSuite implementation.
Useful? React with 👍 / 👎.
| LOG.error("Could not parse %s", filename) | ||
| ``` | ||
|
|
||
| `__name__` for an addon resolves to the addon's `id` (e.g. `"MyAddon.myaddon"`), so the logger inherits the addon's name naturally — useful for per-logger filtering below. |
There was a problem hiding this comment.
Derive debug logger names from the imported module
The registration id does not determine Python's __name__: Gramps imports a flat addon's fname from the temporarily added addon directory, so DescendantSpaceTree.py, for example, logs as DescendantSpaceTree even though its registration id is descendantspacetree; nested packages use their actual package path. Consequently the suggested --debug=MyAddon.myaddon filters commonly match no logger and leave DEBUG output hidden.
Useful? React with 👍 / 👎.
| - **MUST**: open the PR body with a **`**User impact:**`** line (before Root cause), then structure it **Summary / What to look at / Root cause / Fix / Verification**, citing `path:lines` on the branch the PR targets in the Verification "Checked" line (the #106 format). | ||
| - **MUST**: when the PR modifies an addon, **call out its current maintainer** — add an `## Affected addon` section to the PR body that **@-mentions the addon's current maintainer**, a heads-up so they are *aware* of the change and don't miss it. This is awareness, not attribution. "Current maintainer" = the addon's `.gpr.py` `maintainers` field when declared, otherwise its `authors` (an addon with no separate maintainer is maintained by its original author — Doug's "original developer (or contributors)" and Nick's "current maintainer" are the same role). The `.gpr.py` records names/emails, not GitHub handles — resolve a handle best-effort from the declared email so the mention notifies, and name the person when no handle resolves. (Raised on addons-source PR #946 — Doug Blank: *"otherwise I could miss fixes to my addons"*; Nick Hall: *"mention the current maintainer if one exists."*) | ||
| - **OPTIONAL**: reference the Mantis bug in the PR body **when one exists** — a Mantis reference is optional for addons-source, since many addon fixes have no Mantis ticket (they're tracked as fork GitHub issues, or are ticketless). addons-source also does not use the `Fixes #NNNN` commit-message trailer at all — that is the core convention; see the [Core Development — Rules](https://gramps-project.org/wiki/index.php/Gramps_6.1_Wiki_Manual_-_Core_Development_-_Rules) page. | ||
| - **MUST**: keep upstream-repo cross-references out of PR text and fork issues — reference *other* upstream PRs/issues in **plain text** ("upstream PR 949"), never a GitHub URL or `owner/repo#NNN` cross-ref (it back-links/notifies that thread). The `#nnnn` Mantis reference and the PR's own target are exempt. Authoritative: `docs/INTEGRATION.md` §"No upstream-repo links". |
There was a problem hiding this comment.
Add or replace the cited integration policy
A repository-wide file search and the commit tree contain no docs/INTEGRATION.md, so contributors cannot inspect the section asserted to be authoritative for this unusual MUST-level PR rule. Either include the cited policy in this repository or link to the actual source; otherwise the new normative manual relies on an unverifiable, nonexistent local document.
Useful? React with 👍 / 👎.
0f1011f to
2ec8004
Compare
…ields IsSiblingofNamedSibling crashed with HandleError whenever applied to a person with no recorded parents, because get_family_from_handle(None) raises rather than returning None. RegExpPersonal/RegExpFamily also searched mismatched Name fields (title was listed twice in the personal list, and call name was searched by the family/surname rule instead), so personal search never matched a person's call name and surname search could false-positive on an unrelated title or call name. Adds unit tests covering both regressions plus the general relationship-matching rules, and brings the addon up to date with black/mypy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…wiki page help_url pointed at Addon:AdvancedPersonFilter, a different addon name left over from this addon's origin, instead of Addon:PersonRelationshipFilter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`from PersonRelationshipFilter import (HasName, ...)` breaks when unittest loads this file as `PersonRelationshipFilter.tests.test_...`, because by then the outer `PersonRelationshipFilter` is already a namespace package in sys.modules, so the bare import looks for each name as an attribute of the package instead of importing the PersonRelationshipFilter.py submodule that defines them. Use the fully-qualified `PersonRelationshipFilter.PersonRelationshipFilter` import path, matching the pattern already used in DataEntryGramplet/tests/test_data_entry_gramplet.py. Reported by GaryGriffin in gramps-project#987. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2ec8004 to
38036e5
Compare
Some agentic coding tools look for AGENTS.md rather than CLAUDE.md; keep the two in sync so contributors using either tool see the same repo conventions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Eduard R. <eduralph@users.noreply.github.com>
- Clarify branch model: "release" means major/feature release, not patch release; note master exists but isn't currently used for addon work. - Note LANGUAGE=en_US.UTF-8 is no longer required on Gramps v6.0+, and mention checking out the matching branch in the core checkout. - Soften Black formatting guidance to reflect it's not currently required. - Note that strings already translated in Gramps core are excluded from the Weblate Addons component. - Add maintainers/maintainers_email fields to the .gpr.py example.
Per hgohel's suggestion on PR gramps-project#991 (comment 5026237564) and follow-up discussion, clarify the distinction between MSYS2 (POSIX-like shell, safe to run make.py/tests in) and native Windows cmd/PowerShell (unverified tooling) so agents know what they can and can't run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
d2dabee to
b23ca4d
Compare
058801a to
99535fa
Compare
99535fa to
3ca5d85
Compare
26efb2c to
9ad39ee
Compare
…ckend Lets Gramps open a gramps-web-api server (e.g. gramps-connect, Gramps Web) as a regular family tree -- read and write, no export/import step. Subclasses the stock SQLite DBAPI backend rather than reimplementing DbReadBase/DbWriteBase, and keeps a local mirror in sync incrementally via the server's transaction-history endpoint; local edits push back through transaction_commit(). Credentials come from a single GRAMPS_WEB_API_KEY env var (a non-expiring refresh token) rather than a login dialog, which also makes the same webapi_client.py usable as a bare SDK outside Gramps. Status UNSTABLE: no conflict handling (writes are last-write-wins by design, not yet), no undo/redo integration, no media sync. Verified end-to-end against a live gramps-web-api server, including live use from Gramps desktop itself, but no automated test suite yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…radeoff Explains that GRAMPS_WEB_API_KEY carries a standard, non-expiring refresh token from the server's normal login flow rather than a scoped/revocable personal access token, so a leaked key is as damaging as a leaked password until it's changed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Cover webapi_client.WebApiHandler (token codec, JWT decoding, auth flows, 429/401 retry and API-prefix fallback, transaction-history/push request shape) and grampswebapidb.WebApiDB (transaction_to_json, _apply_change, _sync_from_server pagination, transaction_commit ordering and error handling). No real server or SQLite file is needed; urlopen and the DBAPI/SQLite base are stubbed throughout. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Drop force=1 from POST /transactions/ so the server's old-data-mismatch check actually runs. A rejected push now raises WebApiPushConflict (webapi_client.py), which transaction_commit() catches separately from generic connection errors: it logs a distinct warning and resyncs from the server so the local mirror stops showing an edit the server never accepted, rather than drifting silently. webapi_client.py's docstring also now notes it's a hand-synced vendored copy of the standalone gramps-web-api-client package. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…y CLI Documents the standalone package's CLI as the primary way to mint a GRAMPS_WEB_API_KEY, with the addon's own vendored WebApiHandler.mint_api_key() as the equivalent no-extra-dependency fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-api-client rename The standalone client package was renamed (gramps_web_api_client -> gramps_api_client, new checkout at ~/gramps/gramps-api-client). Updated webapi_client.py's docstring and README.md accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ncing it Gramps core's DbGenericUndo._undo()/_redo() revert the local mirror via low-level _txn_begin()/undo_data()/_txn_commit() calls that never go through transaction_commit(), so a local Undo/Redo previously left the server unchanged with no error at all -- worse than a push conflict, since nothing was even logged. WebApiDB now overrides undo()/redo(): both grab the relevant DbTxn off DbGenericUndo's queue before delegating to super(), rebuild its payload with the existing transaction_to_json(), and push it. Undo sends it to POST /transactions/?undo=1, where gramps-web-api reverses it itself (reverse_transaction()); redo just re-pushes the original forward payload, same as an ordinary commit. Both share the same conflict-detection/resync path as transaction_commit(), factored out into _push_payload(). Verified end-to-end against a live server: add a person, undo (a fresh mirror sync confirms the server no longer has it), redo (confirms it's back). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- grampswebapidb.py: use the documented try/except get_addon_translator fallback (glocale.translation directly if the addon has no locale/ translations yet) instead of a bare glocale.translation.gettext. - grampswebapidb.gpr.py: add help_url pointing at the addon's wiki page. - MANIFEST: include README.md in the built .addon.tgz -- it documents the GRAMPS_WEB_API_KEY security tradeoff, not just dev notes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ommit gap A batch=True commit (any bulk import, merge, or tool run through gramps-web-api) leaves an empty-changes marker in the transaction history instead of per-object entries, since DBAPI's commit/remove methods skip the undo-log call for batch transactions. _sync_from_server() had no way to detect this and silently missed everything the batch commit did -- confirmed live: importing example.gramps's 2157 people into a synced tree left the local mirror stuck at its pre-import count indefinitely, no matter how often it resynced. _sync_from_server() now treats an empty-changes transaction as a signal, not a no-op, and falls back to a new _full_resync(): download the server's current Gramps XML export and reimport it (via the same stock ImportXml the batch commit itself used) into a freshly wiped local mirror.
…live load() now schedules a periodic re-sync (GLib.timeout_add_seconds), matching gramps-connect's own browser-client poll against the same endpoint, so edits made from another client show up here without closing and reopening the tree; close() cancels the pending timeout. Since _sync_from_server()'s replay runs in a batch=True DbTxn, DBAPI's own add/update/delete signals never fire for it -- _emit_change_signals() reproduces them by hand so already-open views refresh the same way they would for a local edit, and _full_resync() now calls request_rebuild() for the same reason on its wipe-and-reimport path.
Previously a rejected push (someone else changed the object server-side since the local mirror last synced) just resynced from the server and dropped the local edit. Now it retries the edit on top of the resynced data, merging list-valued fields (notes, citations, events, ...) via the object's own merge() -- the same logic behind Gramps' Merge People/ Family/... tools, ported from GrampsWebSync's diffhandler.py -- instead of one side blindly overwriting the other. A second conflict on the retry itself is logged and dropped rather than retried again.
…report sync progress Nothing tied a local mirror to a particular GRAMPS_WEB_API_KEY account: switching the env var to a different server/tree while reopening the same Family Tree left the mirror silently mixing stale data from the old account with whatever the incremental sync happened to pull from the new one. load() now calls _check_identity(), which requires the Family Tree's own name to be "<username>@<host>" for whoever the key currently authenticates as (normalized through the same character substitution Gramps' own Family Tree Manager applies when renaming a tree, since a hostname's dots can never survive that GUI round-trip intact) -- a mismatch fails DbConnectionError instead of loading. load()'s callback parameter (the same hook Gramps wires to its real progress bar) is now threaded through to _sync_from_server(), which reports page-by-page percent-complete using the server's X-Total-Count, and to _full_resync(), which gets coarse 0/100 markers since ImportXml has no internal step reporting to forward finer progress from. A slow initial catch-up now shows real progress instead of Gramps looking hung. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Core sync/write-through/undo-redo/conflict-detection paths are verified against a live server and covered by tests; remaining gaps (media file sync, scalar-field conflict resolution) are documented limitations rather than missing basics. template.pot was stale, missing strings added by the last commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
152d0b1 to
37ff2e0
Compare
…e its Family Tree Minting a key previously required a shell with gramps-api-client installed, or hand-writing the three lines of Python from the addon's README. Tools -> Utilities -> Generate Gramps Web API key now turns a server URL/username/password into a key from a form, sets it in the running Gramps process's environment so a WebApiDB-backed tree can be opened without restarting, and offers a follow-up button to create a correctly-named, correctly-backed Family Tree for that key via the same CLIDbManager API Gramps' own Family Tree Manager uses. Also tightens grampswebapidb.py's own error handling: load()'s initial sync is now wrapped the same way _check_identity() already was, so a failure there raises the addon's DbConnectionError instead of escaping as a raw HTTPError that Gramps' generic dbloader dialog shows unhelpfully; and HTTP 403s specifically get a message calling out that it's a permissions problem with the authenticated account, not a URL/credentials one.
Gramps only populates the Tools menu once some Family Tree is open, so the "Generate Gramps Web API key" tool still needs one open (any backend) to be reachable, even though it doesn't touch that tree's data. README.md and the tool's own module docstring said otherwise. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… create its Family Tree#1012 Also update template.pot
9d23d2b to
688ab75
Compare
688ab75 to
5f5d33b
Compare
Automated nightly sync from
gramps-project/addons-source@maintenance/gramps60. Generated by .github/workflows/upstream-sync.yml on the testbed.