docs: document Google Workspace connector 15.9 behaviour changes in all seven languages - #515
Open
marevol wants to merge 1 commit into
Open
docs: document Google Workspace connector 15.9 behaviour changes in all seven languages#515marevol wants to merge 1 commit into
marevol wants to merge 1 commit into
Conversation
…ll seven languages The fess-ds-gsuite connector was substantially reworked for 15.9. The ds-gsuite.rst page still described the pre-15.9 connector, including a default OAuth scope and a default crawl behaviour that no longer exist, so an operator following it would hit a hard startup failure with no explanation. Rewrite the page in en/ja/de/es/fr/ko/zh-cn against the connector source. Breaking changes now documented up front, in a new "Changes in 15.9" section with a warning directive: - crawl_target defaults to shared_drives and requires impersonate_user for every value except legacy. An existing configuration upgraded unchanged fails at startup with a DataStoreException. crawl_target=legacy is the escape hatch. - The default OAuth scope narrowed from .../auth/drive to .../auth/drive.readonly. The domain-wide delegation entry in the Google Workspace admin console lists scopes explicitly, so it must be updated. - crawl_target=users and both additionally require .../auth/admin.directory.user.readonly, validated at startup. - The indexed URL is now webViewLink instead of the download link; a full re-crawl is needed. - default_permissions is a fallback, not an addition, and a document whose ACL resolves to nothing is skipped rather than indexed with no roles. - Link-only sharing (allowFileDiscovery=false on domain/anyone) no longer grants a search role. - fields no longer defaults to "*" but to an explicit projection; a field outside it reads null in the crawl script. - Google Docs export as Markdown and Sheets as TSV; re-crawl required. - refresh_token_interval is ignored, with a warning. - Forms and Sites are indexed as metadata only instead of erroring. New sections added in every language: Crawl Target (legacy/shared_drives/ users/both, the startup validation, and why a service account's own view reaches so little), Incremental Crawling (change feed per scope, delete_old_docs forced to false, when tokens are discarded), Rate Limiting and Retries, Text Extraction of Native Google Types, a Limitations section, and troubleshooting entries for the startup failure and for skipped documents. The parameter table is split into a Parameter List and an Advanced Parameters table covering the full set: impersonate_user, crawl_target, scopes, user_query, incremental, domain_permission_format, thread_pool_timeout_seconds, page_size, permission_page_size, max_retries, retry_initial_interval_ms, max_backoff_ms, proxy_username and proxy_password. Notes record that credential parameters are stripped from the script context and that start_page_tokens and crawl_signature are written back by the connector and must be left alone. Parameter names, defaults and all configuration and script samples are identical across the seven languages; only the prose is translated. The Spanish page was already missing sections the other six have (the usage-example group, four troubleshooting entries and the permissions chapter). That gap predates this change and is left as-is; the new 15.9 sections are added to it like everywhere else. All seven files parse cleanly under docutils with the Sphinx-only :doc: role and the |Fess| substitution stubbed. A real Sphinx build was not run: conf/ext.py is Python 2 source and cannot be imported by the Python 3 toolchain available here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Revises the Google Workspace connector page for 15.9 in all seven languages. The connector was
substantially reworked and several of the changes break existing configurations at startup, so the
documentation needed to change with it.
Only the
15.9(development) tree is touched. No released version tree is modified.Why this is a large diff
15.9 changes the connector's default behaviour in ways an operator must know about before upgrading:
crawl_targetdefaults toshared_drivesand requiresimpersonate_user, so an unchangedconfiguration fails at startup. The
crawl_target=legacyescape hatch is documented alongside it.drive.readonly, andcrawl_target=users/bothadditionallyrequire
admin.directory.user.readonly— both of which must also be authorised in the GoogleWorkspace admin console's domain-wide delegation entry.
default_permissionsbecame a fallback rather than an addition; link-only sharing no longer grantsa search role; a document whose ACL resolves to nothing is now skipped rather than indexed with no
roles at all.
fieldsdefault, and the Google Docs/Sheets export formats all changed, eachrequiring a re-crawl.
New sections cover
crawl_target, incremental crawling, rate limiting and retries, text extraction,and a Limitations section.
Accuracy
Everything was checked against the connector source rather than against the plan, which corrected
several points:
url_filteris not a parameter. It exists as an internal constant but is never read from theconfiguration, so setting it does nothing.
include_pattern/exclude_patternare the realfiltering knobs, and are documented instead.
corporais inert under the default crawl target — onlylegacyreads it.spacesapplies tolegacyandusers, notlegacyalone.page_sizeandpermission_page_sizeare clamped, not merely defaulted; a larger value issilently reduced because the API rejects anything above the cap.
listings are skipped and reported, but drive enumeration, user enumeration and permission listing
still fail the crawl deliberately, so a crawl that enumerated nothing cannot report success.
exportFormatsmap, so Markdown and TSV apply only whereDrive offers them; the documented fallback is plain text and CSV.
start_page_tokensandcrawl_signatureare written back by the plugin into the handler parameterfield, where they look user-editable. The pages now say to leave them alone.
The proxy authentication limitation is documented rather than glossed: the credentials travel as a
Proxy-Authorizationrequest header, which does not authenticate an HTTPSCONNECTtunnel — and allGoogle API traffic is HTTPS. The JVM-level workaround is given.
Validation
All seven files parse cleanly under
docutilsat INFO level and above, with the Sphinx-only:doc:role and the
|Fess|substitution stubbed. That covers heading underline widths, directive syntax,list-tablestructure, inline markup and internal target resolution. Two pre-existing warning setswere fixed in passing (30 in
fr, 5 inko, all short underlines); both are now zero.Parity was checked by script: the parameter-name set is identical across all seven files, and every
internal section reference resolves in each.
A full Sphinx build was not run.
conf/ext.pyis Python 2 source and fails to import under theavailable Python 3.12 / Sphinx 9.1. That is unrelated to this change, but it means the HTML output is
unverified here.
Pre-existing gap in the Spanish page — not introduced here
es/15.9/config/datastore/ds-gsuite.rstis missing ten sections that the other six languageshave, and has been for some time:
and Large Number of Files.
Every genuinely new 15.9 section was added to Spanish along with the rest. The pre-existing gap was
deliberately left alone rather than folded into this change. One consequence worth noting: because
Spanish has no permissions chapter, the Drive-permission-to-Fess-role mapping table appears in six
languages but not Spanish, which carries the same semantics in its changes table, the
default_permissionsrow and Limitations instead.Worth a separate issue.