fix: validate custom property name charset#27808
Conversation
github-actions
Bot
added
backend
safe to test
✅ TypeScript Types Auto-UpdatedThe generated TypeScript types have been automatically updated based on JSON schema changes in this PR. |
There was a problem hiding this comment.
Pull request overview
This PR tightens custom property name validation to prevent a small set of characters that break downstream parsers, by enforcing a stricter JSON Schema pattern plus server-side validation and integration tests.
Changes:
- Added a
customPropertyNamedefinition inbasic.jsonwith a restricted character set and max length. - Updated
customProperty.jsonto validatenameagainstcustomPropertyNameinstead ofentityName. - Added Java-side validation in
TypeRepositoryand integration tests inTypeResourceITfor allowed/disallowed names, leading character, and length rules.
Reviewed changes
Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| openmetadata-spec/src/main/resources/json/schema/type/customProperty.json | Switches name validation to the new customPropertyName definition and updates the field description to document the new rules. |
| openmetadata-spec/src/main/resources/json/schema/type/basic.json | Introduces the reusable customPropertyName schema definition (pattern + length). |
| openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TypeRepository.java | Adds defense-in-depth validation for custom property names during add/update operations. |
| openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/TypeResourceIT.java | Adds integration coverage for allowed/disallowed character sets, leading character rules, and max length. |
| @@ -228,6 +234,26 @@ private void validateProperty(CustomProperty customProperty) { | |||
| } | |||
| } | |||
|
|
|||
| private static void validateCustomPropertyName(String name) { | |||
| if (name == null || name.isEmpty()) { | |||
| throw new IllegalArgumentException("Custom property name must not be empty"); | |||
| } | |||
| if (name.length() > CUSTOM_PROPERTY_NAME_MAX_LENGTH) { | |||
| throw new IllegalArgumentException( | |||
| String.format( | |||
| "Custom property name '%s' exceeds maximum length of %d characters", | |||
| name, CUSTOM_PROPERTY_NAME_MAX_LENGTH)); | |||
| } | |||
| if (!CUSTOM_PROPERTY_NAME_PATTERN.matcher(name).matches()) { | |||
| throw new IllegalArgumentException( | |||
| String.format( | |||
| "Invalid custom property name '%s'. Name must start with an alphanumeric character " | |||
| + "and may contain only: alphanumeric, _ - . / & %% # @ ! , ; = | ' space ( ) < > [ ] { }. " | |||
| + "The following characters are not allowed: \" : ^ $", | |||
| name)); | |||
| } | |||
| } | |||
There was a problem hiding this comment.
validateCustomPropertyName is only invoked from addCustomProperty() via validateProperty(CustomProperty). However, Type supports PATCH updates (TypeResource PATCH /{id} and PATCH /name/{fqn}), and the generic patch flow only calls TypeRepository.prepare() (which currently only validates property types via TypeRegistry.validateCustomProperties) before persisting changes. This means a JSON Patch that adds/updates customProperties can still persist names containing ", :, ^, $, etc., bypassing the new defense-in-depth validation.
To fully enforce the new constraint on all write paths, also validate custom property names when Type entities are prepared/updated (e.g., iterate over type.getCustomProperties() in prepare() and call validateCustomPropertyName, or validate in TypeUpdater.updateCustomProperties() before storing added properties).
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
Jest test CoverageUI tests summary
|
🔴 Playwright Results — 2 failure(s), 12 flaky✅ 3982 passed · ❌ 2 failed · 🟡 12 flaky · ⏭️ 98 skipped
Genuine Failures (failed on all attempts)❌
|
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
✅ TypeScript Types Auto-UpdatedThe generated TypeScript types have been automatically updated based on JSON schema changes in this PR. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 7 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TypeRepository.java:228
validateCustomPropertyNameis only invoked fromvalidateProperty(CustomProperty), which is used by the/types/{id}add/update-property endpoint. Other write paths (notablyTypeUpdater.updateCustomProperties()→storeCustomProperty()during PUT/PATCH of aType) can still persist custom properties without any name validation, allowing clients to bypass this restriction via JSON Patch/PUT. Consider validating allupdated.getCustomProperties()entries (e.g., callvalidateCustomPropertyName/validatePropertyforaddeditems, or enforce it inprepare(Type, boolean)for every custom property) so the rule cannot be bypassed.
private void validateProperty(CustomProperty customProperty) {
validateCustomPropertyName(customProperty.getName());
switch (customProperty.getPropertyType().getName()) {
case "enum" -> validateEnumConfig(customProperty.getCustomPropertyConfig());
case "table-cp" -> validateTableTypeConfig(customProperty.getCustomPropertyConfig());
case "date-cp" -> validateDateFormat(
| "name": { | ||
| "description": "Name of the entity property. Note a property name must be unique for an entity. Property name must follow camelCase naming adopted by openMetadata - must start with lower case with no space, underscore, or dots.", | ||
| "$ref": "../type/basic.json#/definitions/entityName" | ||
| "description": "Name of the entity property. Must be unique for an entity. Allowed characters: alphanumeric, _ - . / & % # @ ! , ; = | ' + ? * ~ ` space ( ) < > [ ] { }. Must start with an alphanumeric character. Disallowed: \" : ^ $ \\.", |
There was a problem hiding this comment.
The name.description text ends with \\. which renders as \. (backslash + dot) and is confusing/inaccurate (dot is allowed by the pattern). If the intent is to list \ as a disallowed character, update the description to show a literal backslash without the trailing . and keep the allowed/disallowed lists consistent with basic.json#/definitions/customPropertyName.
| "description": "Name of the entity property. Must be unique for an entity. Allowed characters: alphanumeric, _ - . / & % # @ ! , ; = | ' + ? * ~ ` space ( ) < > [ ] { }. Must start with an alphanumeric character. Disallowed: \" : ^ $ \\.", | |
| "description": "Name of the entity property. Must be unique for an entity. Allowed characters: alphanumeric, _ - . / & % # @ ! , ; = | ' + ? * ~ ` space ( ) < > [ ] { }. Must start with an alphanumeric character. Disallowed: \" : ^ $ \\", |
| "customPropertyName": { | ||
| "description": "Name of a custom property. Allowed characters: alphanumeric, _ - . / & % # @ ! , ; = | ' + ? * ~ ` space ( ) < > [ ] { }. Must start with an alphanumeric character. Disallowed characters: \" : ^ $ \\", | ||
| "type": "string", | ||
| "minLength": 1, | ||
| "maxLength": 256, | ||
| "pattern": "^[A-Za-z0-9][A-Za-z0-9 _\\-.,;/&%#@!'(){}\\[\\]<>|=+?*~`]*$" | ||
| }, |
There was a problem hiding this comment.
The new customPropertyName regex here allows additional characters (+ ? * ~ �backtick�) and also implicitly disallows backslash (\\), but the PR description and some generated docs elsewhere mention only four disallowed characters and omit these allowed ones. Please either align the pattern with the intended charset, or update the PR/docs to reflect the actual rule (including whether \\ is intentionally disallowed).
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
Tighten custom property name validation to block characters that break
downstream parsers, with verified empirical reproduction:
- `"` causes HTTP 500 on PUT /metadata/types/{id}
- `:` breaks CSV import — exporter writes `key:value;key:value`, importer
splits at first colon, treats prefix as the field name
- `^` breaks OpenSearch query when the name is in
searchSettings.searchFields — Lucene reads `^` as the boost separator
in `field^boost`
- `$` breaks CSV import via java.util.regex.Matcher.replaceAll which
interprets `$<letter>` as a backreference
Adds a `customPropertyName` definition in basic.json and switches
customProperty.json to reference it. Adds a defensive regex check in
TypeRepository.validateProperty so the API returns 400 with a clear
error message even if schema validation is bypassed.
Tests cover allowed-charset acceptance, the four blocked characters,
leading-character validation, max-length enforcement, and unbalanced
brackets.
|
The Java checkstyle failed. Please run You can install the pre-commit hooks with |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 35 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
openmetadata-ui/src/main/resources/ui/src/components/common/SanitizedInput/SanitizedInput.tsx:32
handleChangenow depends onshouldSanitize, but the callback memoization still only tracksonChange. If a parent togglesshouldSanitizeafter the component mounts, this handler will keep using the old mode and sanitize when it should not (or vice versa).
const handleChange = useCallback(
(e: ChangeEvent<HTMLInputElement>) => {
const sanitizedValue = shouldSanitize
? getSanitizeContent(e.target.value)
: e.target.value;
if (onChange) {
onChange({ ...e, target: { ...e.target, value: sanitizedValue } });
}
},
[onChange]
| max: 128, | ||
| message: t('message.entity-size-in-between', { | ||
| entity: t('label.name'), | ||
| min: 1, | ||
| max: 128, |
| `^${optionTitle.replaceAll(/[.*+?^${}()|[\]\\]/g, '\\$&')}$` | ||
| ), | ||
| }) | ||
| .locator(`[title="${optionTitle}"]`) |
| test('should show error when name exceeds 128 characters', async ({ | ||
| page, | ||
| }) => { | ||
| await page.fill(nameInput, INVALID_NAMES.MAX_LENGTH); | ||
|
|
||
| await expect(page.locator(nameError)).toContainText( | ||
| NAME_MAX_LENGTH_VALIDATION_ERROR | ||
| ); |
…sertions Schema is the single source of truth: jsonschema2pojo emits @pattern + @SiZe on CustomProperty.name from basic.json#/definitions/customPropertyName, and @Valid on TypeResource.addOrUpdateProperty enforces them at the HTTP boundary. The hand-written Pattern constant, validateCustomPropertyName, and the schema-vs-Java sync test were duplicating that rule and could never reach the HTTP user (Bean Validation always fires first via @Valid). Tighten the new TypeResourceIT cases from assertThrows(Exception.class) to assertThrows(InvalidRequestException.class) so a regression to a different exception type or status code fails loudly.
sonika-shah
force-pushed
the
fix/custom-property-name-charset-validation
branch
from
35d06c2 to
3c92e1b
Compare
Code Review ✅ Approved 1 resolved / 1 findingsTightens custom property name validation to prevent downstream parser failures by blocking problematic characters like ", :, ^, $, and . Integration tests were refined to assert 400-level error responses, and redundant Java validator tests were removed. ✅ 1 resolved✅ Bug: assertEquals arguments are swapped (expected vs actual)
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 33 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
openmetadata-ui/src/main/resources/ui/src/components/common/SanitizedInput/SanitizedInput.tsx:32
shouldSanitizeis now part of this component's public API, buthandleChangestill only depends ononChange. If a caller togglesshouldSanitizewithout remounting, the callback keeps using the old mode and sanitizes (or skips sanitizing) incorrectly until the component is recreated.
const handleChange = useCallback(
(e: ChangeEvent<HTMLInputElement>) => {
const sanitizedValue = shouldSanitize
? getSanitizeContent(e.target.value)
: e.target.value;
if (onChange) {
onChange({ ...e, target: { ...e.target, value: sanitizedValue } });
}
},
[onChange]
| max: 128, | ||
| message: t('message.entity-size-in-between', { | ||
| entity: t('label.name'), | ||
| min: 1, | ||
| max: 128, |
| type: FieldTypes.TEXT_MUI, | ||
| props: { | ||
| 'data-testid': 'display-name', | ||
| shouldSanitize: false, |
| type: FieldTypes.TEXT, | ||
| props: { | ||
| 'data-testid': 'display-name', | ||
| shouldSanitize: false, |
| test('should show error when name exceeds 128 characters', async ({ | ||
| page, | ||
| }) => { | ||
| await page.fill(nameInput, INVALID_NAMES.MAX_LENGTH); | ||
|
|
||
| await expect(page.locator(nameError)).toContainText( | ||
| NAME_MAX_LENGTH_VALIDATION_ERROR | ||
| ); |
| test('should accept a valid name with allowed special characters', async ({ | ||
| page, | ||
| }) => { | ||
| await page.fill(nameInput, "valid Name_-./&%#@!,;=|'()<>[]{}"); | ||
|
|
| "description": "Name of the entity property. Must be unique for an entity. Allowed characters: alphanumeric, _ - . / & % # @ ! , ; = | ' + ? * ~ ` space ( ) < > [ ] { }. Must start with an alphanumeric character. Disallowed: \" : ^ $ \\.", | ||
| "$ref": "../type/basic.json#/definitions/customPropertyName" |
|
|
Fixes #27699
Summary
Tighten custom property name validation to block characters that empirically break downstream parsers. Adds a new
customPropertyNameschema definition, defense-in-depth Java validation inTypeRepository, a schema-vs-Java consistency test that prevents drift between the two, and integration tests covering allowed/blocked charsets, leading-character rules, length, and unbalanced brackets.Why
Custom property names previously inherited the very permissive
entityNamepattern (^((?!::).)*$), which only blocks::. A matrix run on 1.13 across 24 special chars × 17 property types × 4 operations identified five characters that break specific operations regardless of property type:"PUT /metadata/types/{id}:key:value;key:value; importer splits at first:, treats prefix as the field name (Unknown custom field: <prefix>)^searchSettings.searchFields, OpenSearch reads^as the boost separator in thefield^boostgrammar (x_content_parse_exception [bool] failed to parse field [should])$Matcher.replaceAllinterprets$<letter>as a regex backreference (Illegal group reference) when the value contains certain characters\entityReference/entityReferenceListtypes (export emits double-backslash, import un-escapes to single, key lookup fails)All other tested characters (alphanumeric,
_,-,.,/,&,%,#,@,!,,,;,=,|,',+,?,*,~,`, space,(,),<,>,[,],{,}) round-trip cleanly across CREATE / PATCH / CSV (real, non-dry-run) / Search across all 17 property types, including multi-property CSV roundtrips that exercise the;and,CSV-quoting layer.Final charsets
A-Z a-z 0-9 _ - . / & % # @ ! , ; = | ' + ? * ~space ( ) < > [ ] { }`" : ^ $ \Changes
openmetadata-spec/.../type/basic.json— addscustomPropertyNamedefinition with regex^[A-Za-z0-9][A-Za-z0-9 _\-.,;/&%#@!'(){}\[\]<>|=+?*~]*$`, max length 256.openmetadata-spec/.../type/customProperty.json—namenow$refscustomPropertyNameinstead ofentityName.openmetadata-service/.../jdbi3/TypeRepository.java— addsvalidateCustomPropertyName(String)called fromvalidateProperty(CustomProperty)so the API returns 400 with a clear, human-readable error message even if schema validation is bypassed (e.g. internal callers that don't go through the@Valid-protected resource layer).openmetadata-service/.../jdbi3/TypeRepositoryTest.java(new) — schema-vs-Java consistency tests that loadbasic.jsonat runtime and assert the pattern andmaxLengthmatch theTypeRepositoryconstants. Drift in either direction fails CI with a message naming both files.Why two layers + a sync test (instead of one)
Each layer earns its keep:
@ValidBean Validation flow.@Patternviolation (which dumps the raw regex), and runs even on code paths that buildCustomPropertyprogrammatically without crossing the HTTP boundary.Backwards compatibility
Validator runs only on writes. Existing custom properties with names containing the now-blocked characters continue to read normally — only new creates / updates are rejected.
Test plan
Unit tests in
openmetadata-service(TypeRepositoryTest)customPropertyNamePatternMatchesSchema— schema regex matches JavaPatterncustomPropertyNameMaxLengthMatchesSchema— schemamaxLengthmatches Java constantcustomPropertyNameDefinitionIsRequiredString— definition shape sanity checkIntegration tests in
openmetadata-integration-tests(TypeResourceIT)test_customPropertyNameAllowedCharacters_succeeds— 31 names spanning every safe character (including+,?,*,~,`)test_customPropertyNameDisallowedCharacters_fails— verifies",:,^,$,\are rejectedtest_customPropertyNameMustStartWithAlphanumeric_fails— leading char rule for_,-,.,(,<test_customPropertyNameTooLong_fails— > 256 character names rejectedtest_customPropertyNameUnbalancedBrackets_succeeds—(,),<,>,[,],{,}accepted unbalanced (no closure required)🤖 Generated with Claude Code
Summary by Gitar
CUSTOM_PROPERTY_NAME_REGEXintoAddCustomPropertycomponents and updated regex constants.SanitizedInputandMUITextFieldto optionally bypass sanitization for custom property inputs.custom-property-name-validationerror messages across all supported languages to reflect the new charset restrictions.This will update automatically on new commits.