[7418] plugin - serializers/retrievers/validators

studio-ui/docs/type-builder-forms-engine-plugins.md

@@ -2,7 +2,7 @@
 
 > Companion to [`type-builder-forms-engine.md`](type-builder-forms-engine.md). Read this before designing or changing dynamic controls, data sources, FE form controllers, plugin loading, or TB plugin discovery.
 
-Last updated: 2026-08-03
+Last updated: 2026-08-07
 
 ## 1. Why plugins are part of the core design
 
@@ -282,21 +282,28 @@ export default {
 	controls: {
 		'my-custom-picker': {
 			Component: MyPicker,
-			dataSourceBindings: [{ propertyName: 'itemManager', interfaces: ['item'], selection: 'multi' }]
+			dataSourceBindings: [{ propertyName: 'itemManager', interfaces: ['item'], selection: 'multi' }],
+			valueRetriever: (raw, field) => /* XML → form value */,
+			valueSerializer: (field, value) => /* form value → XML-ready shape */,
+			validator: (field, value, messages, meta) => /* type-specific; push messages; return bool */
 		}
 	}
 };
 ```
 
-Eager alternative for UMD loaders: `craftercms.formsEngine.controls.registerDataSourceBindings(type, bindings)` (bindings only; prefer `descriptor.controls` for the component).
+Lookup order for IO and validators: built-in `valueRetrieverLookup` / `valueSerializersLookup` / `validatorsMap` first (`Object.hasOwn`), then plugin contribution for `field.type`. Required/empty checks stay in `validateFieldValue` (host). FE preloads all `field.properties.plugin` locators before form value parse and again before save (`preloadControlPluginsForFields`) so custom retrievers/serializers/validators are registered in time.
 
-Example: `samples/fe2-control-plugin.example.mjs`.
+Eager alternative for UMD loaders: `craftercms.formsEngine.controls.registerDataSourceBindings(type, bindings)` (bindings only; prefer `descriptor.controls` for the component). Host also exposes `getValueRetriever` / `getValueSerializer` / `getValidator` for inspection.
+
+**Field chrome.** Plugin controls are rendered without a wrapper, so validation state is invisible unless the control renders it. `craftercms.formsEngine.controls.FormsEngineField` is the same component built-in controls use — it reads the field's validity atom and renders the label, required/invalid indicator, validator messages, field menu, and inheritance notice:
+
+```js
+const { FormsEngineField } = globalThis.craftercms?.formsEngine?.controls ?? {};
+// <FormsEngineField field={field} htmlFor={id}>{yourInput}</FormsEngineField>
+```
 
 Missing for a complete extension contract:
 
-- plugin-supplied value retriever;
-- serializer;
-- validators;
 - additional XML fields/attributes;
 - migrations/versioning;
 - optional descriptor for TB property editing (should mirror `dataSourceBindings` for the DS property UI);
@@ -459,13 +466,16 @@ interface PluginDescriptor {
 interface ControlPluginContribution {
 	Component: ComponentType<ControlProps>;
 	dataSourceBindings?: DataSourceBinding | readonly DataSourceBinding[];
+	valueRetriever?: ValueRetriever; // optional XML → form value
+	valueSerializer?: ValueSerializer; // optional form value → XML shape
+	validator?: ValidatorFunctionDef; // optional type-specific; required/empty is host-owned
 }
 ```
 
 `registerPlugin` (current):
 
 1. Validates and installs `dataSources` into `dataSourceModuleRegistry` (before committing the plugin id).
-2. Validates and installs `controls` into the control contribution registry + binding registry.
+2. Validates and installs `controls` into the control contribution registry + binding registry (including optional `valueRetriever` / `valueSerializer` / `validator`).
 3. Stores the descriptor (including `utils`) in the plugins map.
 4. Registers widgets, locales, scripts/stylesheets as before.
 5. `widgets` may be omitted (lib / FE-only bundles).

studio-ui/docs/type-builder-forms-engine.md

@@ -302,9 +302,11 @@ Example: `samples/fe2-control-plugin.example.mjs`.
 The runtime control contract is `ControlProps` (`value`, `setValue`, `field`, `contentType`, `readonly`, `autoFocus`, optional `dataSources`). Host helpers:
 
 - `window.craftercms.formsEngine.dataSources` — DS module registry
-- `window.craftercms.formsEngine.controls` — `getControl(type)`, `registerDataSourceBindings`, `getDataSourceBindings`
+- `window.craftercms.formsEngine.controls` — `getControl(type)`, `registerDataSourceBindings`, `getDataSourceBindings`, `getValueRetriever`, `getValueSerializer`, `getValidator`, `FormsEngineField`
 
-This path is functional for plugin metadata already present in a form definition. `customControlMap` exists as an override seam used by Type Builder descriptor forms. Validators, value retrievers, and serializers remain static maps/switches, so a truly novel control may still need more than a React component + bindings export.
+Plugin controls are rendered bare by `ResolvedControlRenderer` (built-ins wrap themselves in `FormsEngineField`). A plugin that wants label, required/invalid styling, and validity messages must wrap its input in the host-provided `FormsEngineField`; otherwise a failing validator only surfaces in the Table of Contents and the save gate.
+
+This path is functional for plugin metadata already present in a form definition. `customControlMap` exists as an override seam used by Type Builder descriptor forms. Plugin controls may supply optional `valueRetriever` / `valueSerializer` / `validator` on `ControlPluginContribution` (built-in static maps still win when they define the type). Required/empty validation stays host-owned in `validateFieldValue`.
 
 Additional current FE control gaps:
 
@@ -633,6 +635,8 @@ Separate **completed design decisions** (`[x]`) from **remaining implementation
 - [x] **Form-controller design** — type-local FE2 ESM (`FormController` hooks), fetch via form_controller API, not `PluginDescriptor`. See §5.9. Implementation still open (below).
 - [x] **Control-plugin ownership** — after `importPlugin()`, ownership is validated against loaded `PluginDescriptor.id` (not the form-definition locator `pluginId`). See `controlPluginLoader.ts`.
 - [x] **Atomic FE plugin registration** — `registerPlugin` preflights all DS + control contributions before any registry commit.
+- [x] **Plugin control valueRetriever / valueSerializer** — optional on `ControlPluginContribution`; registered with the control; FE preloads plugin locators before form parse and save. Walkthrough: [`fe2-plugin-control-io-and-validators.md`](fe2-plugin-control-io-and-validators.md).
+- [x] **Plugin control validator** — optional on `ControlPluginContribution`; looked up via `getFieldValidator` after built-in `validatorsMap`; ToC uses `hasFieldValidator`. Required/empty remains host-owned. Walkthrough: [`fe2-plugin-control-io-and-validators.md`](fe2-plugin-control-io-and-validators.md).
 
 ### Remaining implementation / validation
 
@@ -660,6 +664,10 @@ Separate **completed design decisions** (`[x]`) from **remaining implementation
 
 Keep newest first. One short bullet per meaningful session.
 
+- **2026-08-07** — Expose `FormsEngineField` on `craftercms.formsEngine.controls`. Plugin controls render bare (built-ins wrap themselves), so a failing plugin validator previously showed only in the ToC; wrapping in the host field chrome restores parity (label, invalid styling, validity messages).
+- **2026-08-07** — Plugin control validators: optional `ControlPluginContribution.validator` installed by `registerPlugin`; `getFieldValidator` / `hasFieldValidator` fall back after built-in `validatorsMap`; host `getValidator`; sample rejects angle brackets. Same preload path as IO hooks.
+- **2026-08-07** — Plugin control IO hooks: `ControlPluginContribution.valueRetriever` / `valueSerializer` installed by `registerPlugin`, looked up after built-in maps in `valueRetrievers` / `valueSerializers`. Form bootstrap + save preload plugin locators via `preloadControlPluginsForFields` so hooks exist before parse/serialize. Host: `craftercms.formsEngine.controls.getValueRetriever` / `getValueSerializer`. Sample updated.
+- **2026-08-06** — Controls cleanup (`7418`): retired unused `link-input` / `link-textarea` / `linked-dropdown` from FE2 maps + TB descriptors. Closed the former “non-rendering control-map entries” open item: those three are removed; `disabled` / `internal-name` remain TB catalog ids that remap on insert via `systemFieldsTypesMap` to `checkbox` / `input` (locked field ids). Documented under completed design decisions.
 - **2026-08-03** — Convergence-gap audit reflected in §8: remaining work includes S3/WebDAV stubs, null control-map entries (`disabled` / `internal-name` / `link-input` / `link-textarea`), FE2 RTE plugin parity, and focused compatibility tests. Control-plugin `PluginDescriptor.id` ownership checks and atomic `registerPlugin` preflight were already implemented — recorded under completed design decisions, not left as open gaps.
 - **2026-08-03** — Refined §5.9: all `FormController` hooks may be async (host awaits); clarified on-disk path, form_controller API, and FE2 loader call site (`formControllerLoader` from form bootstrap — not `importPlugin`).
 - **2026-08-03** — Decided FE2 form-controller design (§5.9): keep type-local `form-controller.js` gated by `hasJsController`; load via authenticated form_controller API + ESM Blob import; export `FormController` hooks (`initialize`, `isFieldRelevant`, `onBeforeSave`) — **not** a `PluginDescriptor`. FE1 YUI controllers are incompatible (migrate by rewrite). TB must fix Client-side Controller to edit `form-controller.js` instead of Groovy. Implementation still TODO.

studio-ui/ui/app/src/components/FormsEngine/FormsEngine.tsx

@@ -126,6 +126,7 @@ import SectionAccordion from './components/SectionAccordion';
 import useSaveForm from './lib/useSaveForm';
 import { FormPrepError } from './components/FormPrepError';
 import { createParsedValuesObject } from './lib/valueRetrievers';
+import { preloadControlPluginsForFields } from './lib/controlPluginLoader';
 import { fromString } from '../../utils/xml';
 import { displayWithPendingChangesConfirm } from '../../utils/ui';
 import useActiveUser from '../../hooks/useActiveUser';
@@ -334,6 +335,7 @@ function FormBootstrap(props: FormsEngineProps) {
 	useEffect(() => {
 		// Guard statement: If content types are not loaded, we can't proceed.
 		if (!contentTypesLoaded) return;
+		let disposed = false;
 		// TODO: If props are changed, things can be left off... previous item locked, edits get lost, etc. Not sure how much support for prop changes we should implement.
 		const isChildForm = stackIndex > 0;
 		// In the form stack, the present form being opened would be in the last position [length-1], the parent form state would be on [length-2] if it is nested (e.g. Root => Component(L1) => Repeat(L2)|Component(L2)). Otherwise,the parent should be the root.
@@ -494,43 +496,54 @@ function FormBootstrap(props: FormsEngineProps) {
 				fileName: atom('')
 			});
 			const contentObject = createObjectWithSystemProps(contentType);
-			const values = createParsedValuesObject(
-				contentType.fields,
-				contentObject,
-				contentTypesById,
-				(fieldId, value, isAdditional) => {
-					setFieldAtoms(
-						stableFormContextRef,
-						contentType,
-						contentType.fields,
-						fieldId,
-						atoms,
-						value,
-						{ siteId, contentTypesById },
-						isAdditional
-					);
-				},
-				customControls
-			);
-			const { [XmlKeys.fileName]: _, ...valuesWithoutFileName } = values;
+			const initCreateForm = () => {
+				if (disposed) return;
+				const values = createParsedValuesObject(
+					contentType.fields,
+					contentObject,
+					contentTypesById,
+					(fieldId, value, isAdditional) => {
+						setFieldAtoms(
+							stableFormContextRef,
+							contentType,
+							contentType.fields,
+							fieldId,
+							atoms,
+							value,
+							{ siteId, contentTypesById },
+							isAdditional
+						);
+					},
+					customControls
+				);
+				const { [XmlKeys.fileName]: _, ...valuesWithoutFileName } = values;
 
-			const objectId = contentObject[XmlKeys.modelId] as string;
-			initializeState(atoms, values, {
-				id: objectId,
-				// TODO: Should/could we somehow deduce the target path?
-				path: null,
-				// TODO: Sourcemap? How can we determine what would be inherited by this content? New API?
-				sourceMap: null,
-				pathInSite: processPathMacros({
-					path: create.path,
-					objectId,
-					fullParentPath: '',
-					useUUID: false
-				}),
-				contentType,
-				contentObject,
-				contentXml: buildContentXml(valuesWithoutFileName, contentTypesById)
-			});
+				const objectId = contentObject[XmlKeys.modelId] as string;
+				initializeState(atoms, values, {
+					id: objectId,
+					// TODO: Should/could we somehow deduce the target path?
+					path: null,
+					// TODO: Sourcemap? How can we determine what would be inherited by this content? New API?
+					sourceMap: null,
+					pathInSite: processPathMacros({
+						path: create.path,
+						objectId,
+						fullParentPath: '',
+						useUUID: false
+					}),
+					contentType,
+					contentObject,
+					contentXml: buildContentXml(valuesWithoutFileName, contentTypesById)
+				});
+			};
+			preloadControlPluginsForFields(siteId, contentType.fields)
+				.catch((error) => {
+					console.error('Failed to preload control plugins before create-form value parse.', error);
+				})
+				.then(initCreateForm);
+			return () => {
+				disposed = true;
+			};
 		} /* if (isUpdateMode) */ else {
 			const subscription = fetchUpdateRequirements({
 				siteId,
@@ -570,37 +583,47 @@ function FormBootstrap(props: FormsEngineProps) {
 						expandedStateBySectionId: buildSectionExpandedStateAtoms(requirements.contentType.sections),
 						fileName: createFileNameAtom(requirements.item.path)
 					});
-					const values = createParsedValuesObject(
-						requirements.contentType.fields,
-						requirements.contentObject,
-						effectRefs.current.contentTypesById,
-						(fieldId, value, isAdditional) => {
-							setFieldAtoms(
-								stableFormContextRef,
-								requirements.contentType,
+					preloadControlPluginsForFields(siteId, requirements.contentType.fields)
+						.catch((error) => {
+							console.error('Failed to preload control plugins before edit-form value parse.', error);
+						})
+						.then(() => {
+							if (disposed) return;
+							const values = createParsedValuesObject(
 								requirements.contentType.fields,
-								fieldId,
-								atoms,
-								value,
-								{ siteId, contentTypesById },
-								isAdditional
+								requirements.contentObject,
+								effectRefs.current.contentTypesById,
+								(fieldId, value, isAdditional) => {
+									setFieldAtoms(
+										stableFormContextRef,
+										requirements.contentType,
+										requirements.contentType.fields,
+										fieldId,
+										atoms,
+										value,
+										{ siteId, contentTypesById: effectRefs.current.contentTypesById },
+										isAdditional
+									);
+								},
+								customControls
 							);
-						},
-						customControls
-					);
 
-					initializeState(atoms, values, {
-						id: values[XmlKeys.modelId] as string,
-						path: requirements.item.path,
-						// TODO: Sourcemap? How can we determine what would be inherited by this content? New API?
-						sourceMap: requirements.sourceMap,
-						pathInSite: requirements.pathInSite,
-						contentType: requirements.contentType,
-						contentXml: requirements.contentXml,
-						contentObject: requirements.contentObject
-					});
+							initializeState(atoms, values, {
+								id: values[XmlKeys.modelId] as string,
+								path: requirements.item.path,
+								// TODO: Sourcemap? How can we determine what would be inherited by this content? New API?
+								sourceMap: requirements.sourceMap,
+								pathInSite: requirements.pathInSite,
+								contentType: requirements.contentType,
+								contentXml: requirements.contentXml,
+								contentObject: requirements.contentObject
+							});
+						});
 				});
-			return () => subscription.unsubscribe();
+			return () => {
+				disposed = true;
+				subscription.unsubscribe();
+			};
 		}
 	}, [
 		contentTypesLoaded,
@@ -1231,7 +1254,6 @@ export default FormGuard;
 //  - Where do we put the "config" to determine whether to use new or old form engine?
 //  - Form controller loading and execution
 //  - FOR LATER...
-//    - Allow overriding/extending validators, retrievers, [and maybe] controlMap through plugins
 //    - Inherited non overridable if not in the model
 //    - AI
 //    - Edit template & controller

studio-ui/ui/app/src/components/FormsEngine/components/TableOfContents.tsx

@@ -24,14 +24,13 @@ import SearchBar from '../../SearchBar';
 import { SimpleTreeView } from '@mui/x-tree-view/SimpleTreeView';
 import Box from '@mui/material/Box';
 import { useAtomValue, useSetAtom, useStore as useJotaiStore } from 'jotai/index';
-import { isEmptyValue, isFieldRequired, validatorsMap } from '../lib/validators';
+import { hasFieldValidator, isEmptyValue, isFieldRequired } from '../lib/validators';
 import { atom } from 'jotai';
 import { immutableEmptyArray } from '../../../utils/array';
 import useLoadableAtom from '../lib/useLoadableAtom';
 import Skeleton from '@mui/material/Skeleton';
 import ErrorBoundary from '../../ErrorBoundary';
 import FieldStateIndicator from './FieldStateIndicator';
-import { nnou } from '../../../utils/object';
 
 export interface TableOfContentsProps {
 	containerRef: RefObject<HTMLDivElement>;
@@ -154,7 +153,7 @@ function TreeItemLabel({
 	const validityData = useLoadableAtom(atoms.validationByFieldId[field.id]);
 	const isValid = validityData.state === 'hasData' ? validityData?.data.isValid : true;
 	const isRequired = isFieldRequired(field);
-	const hasValidator = nnou(validatorsMap[field.type]);
+	const hasValidator = hasFieldValidator(field.type);
 	return (
 		<Box display="flex" justifyContent="space-between" alignItems="center">
 			<span>{field.name}</span>