Questions about the GeoNetwork patch for XslUtil

[josegar74]

The code patch contains changes mostly related to the harvesters, but also changes in some other files like XslUtil

dcat-ap/core-geonetwork-patches/patch-core-for-dcat-plugin.patch

Lines 41 to 154 in 9d9d5bc

	--- a/core/src/main/java/org/fao/geonet/util/XslUtil.java
	+++ b/core/src/main/java/org/fao/geonet/util/XslUtil.java
	@@ -23,6 +23,7 @@
	package org.fao.geonet.util;
	+import co.elastic.clients.elasticsearch.core.SearchResponse;
	import co.elastic.clients.elasticsearch.core.search.Hit;
	import com.fasterxml.jackson.core.JsonProcessingException;
	import com.fasterxml.jackson.databind.ObjectMapper;
	@@ -54,6 +55,7 @@ import org.fao.geonet.api.records.attachments.Store;
	import org.fao.geonet.constants.Geonet;
	import org.fao.geonet.domain.*;
	import org.fao.geonet.kernel.*;
	+import org.fao.geonet.kernel.datamanager.IMetadataUtils;
	import org.fao.geonet.kernel.datamanager.base.BaseMetadataUtils;
	import org.fao.geonet.kernel.search.CodeListTranslator;
	import org.fao.geonet.kernel.search.EsSearchManager;
	@@ -61,13 +63,11 @@ import org.fao.geonet.kernel.search.Translator;
	import org.fao.geonet.kernel.security.SecurityProviderConfiguration;
	import org.fao.geonet.kernel.setting.SettingInfo;
	import org.fao.geonet.kernel.setting.SettingManager;
	+import org.fao.geonet.kernel.setting.Settings;
	import org.fao.geonet.kernel.url.UrlChecker;
	import org.fao.geonet.languages.IsoLanguagesMapper;
	import org.fao.geonet.lib.Lib;
	-import org.fao.geonet.repository.IsoLanguageRepository;
	-import org.fao.geonet.repository.SourceRepository;
	-import org.fao.geonet.repository.UiSettingsRepository;
	-import org.fao.geonet.repository.UserRepository;
	+import org.fao.geonet.repository.*;
	import org.fao.geonet.schema.iso19139.ISO19139Namespaces;
	import org.fao.geonet.utils.Log;
	import org.fao.geonet.utils.Xml;
	@@ -1608,4 +1608,80 @@ public final class XslUtil {
	return webAnalyticsConfiguration.getJavascriptCode();
	}
	+
	+ // DCAT Custom
	+ public static String getThesaurusTitleByName(String id) {
	+ ApplicationContext applicationContext = ApplicationContextHolder.get();
	+ ThesaurusManager thesaurusManager = applicationContext.getBean(ThesaurusManager.class);
	+ Thesaurus thesaurus = thesaurusManager.getThesaurusByName(id);
	+ return thesaurus == null ? "" : thesaurus.getTitle();
	+ }
	+
	+ /**
	+ * DCAT custom
	+ * Generate a UUID based off a string for consistent results
	+ *
	+ * @param str
	+ * @return UUID string
	+ */
	+ public static String uuidFromString(String str) {
	+ return UUID.nameUUIDFromBytes(str.getBytes()).toString();
	+ }
	+
	+ // DCAT Custom
	+ public static String getRecordResourceURI(String uuid) {
	+ var searchManager = ApplicationContextHolder.get().getBean(EsSearchManager.class);
	+
	+ try {
	+ Set<String> source = new HashSet<>();
	+ source.add("rdfResourceIdentifier");
	+ SearchResponse response = searchManager.query(String.format("+uuid:%s", uuid), null, source, 0, 1);
	+
	+ if (response.hits().hits().isEmpty()) {
	+ return null;
	+ }
	+ ObjectMapper objectMapper = new ObjectMapper();
	+
	+ Hit h = (Hit) response.hits().hits().get(0);
	+ Object rdfResourceIdentifier = objectMapper.convertValue(h.source(), Map.class).get("rdfResourceIdentifier");
	+ if (rdfResourceIdentifier instanceof String) {
	+ return (String) rdfResourceIdentifier;
	+ } else if (rdfResourceIdentifier instanceof ArrayList) {
	+ return ((ArrayList<String>) rdfResourceIdentifier).get(0);
	+ } else {
	+ throw new Exception("Cannot obtain resource URI from " + rdfResourceIdentifier.toString());
	+ }
	+
	+ } catch (Exception e) {
	+ Log.error(Log.JEEVES, "GET Record resource identifier '" + uuid + "' error: " + e.getMessage(), e);
	+ }
	+ return null;
	+ }
	+
	+ /**
	+ * DCAT Custom
	+ * Get the metadata URI build pattern based on the source catalog
	+ *
	+ * @param uuid
	+ */
	+ public static String getUriPattern(String uuid) {
	+ ApplicationContext context = ApplicationContextHolder.get();
	+ IMetadataUtils metadataUtils = context.getBean(IMetadataUtils.class);
	+
	+ AbstractMetadata metadata = metadataUtils.findOneByUuid(uuid);
	+
	+ if (metadata != null && metadata.getHarvestInfo().isHarvested()) {
	+ HarvesterSettingRepository harvesterSetting = context.getBean(HarvesterSettingRepository.class);
	+ HarvesterSetting uuidSetting = harvesterSetting.findOneByNameAndStoredValueLike("uuid", metadata.getHarvestInfo().getUuid());
	+ if (uuidSetting != null && uuidSetting.getParent() != null) {
	+ List<HarvesterSetting> resourceUriPatternSetting = harvesterSetting.findChildrenByName(uuidSetting.getParent().getId(), "resourceUriPattern");
	+ if (resourceUriPatternSetting.size() > 0 && StringUtils.isNotEmpty(resourceUriPatternSetting.get(0).getValue())) {
	+ return resourceUriPatternSetting.get(0).getValue();
	+ }
	+ }
	+ }
	+
	+ SettingManager sm = context.getBean(SettingManager.class);
	+ return sm.getValue(Settings.SYSTEM_RESOURCE_PREFIX) + "/{resourceType}/{resourceUuid}";
	+ }

The only method with something specific for DCAT seems getRecordResourceURI that uses a custom Elasticsearch field rdfResourceIdentifier.

I see 2 options:

1. Move all the generic methods to Geonetwork opensource XslUtil
2. Create a custom XslUtilDcatAp class in the schema plugin with the method getRecordResourceURI

Or:

1. Create a custom XslUtilDcatAp class with all these methods

And update the related xsl files to use this class.

An example for HNAP:

• https://github.com/metadata101/iso19139.ca.HNAP/blob/4.4.x/src/main/java/ca/gc/schema/iso19139hnap/util/XslUtilHnap.java
• https://github.com/metadata101/iso19139.ca.HNAP/blob/bbe0a5197d6686a1d1d59658a347be77313c0771/src/main/plugin/iso19139.ca.HNAP/layout/layout-custom-fields.xsl#L16
• https://github.com/metadata101/iso19139.ca.HNAP/blob/bbe0a5197d6686a1d1d59658a347be77313c0771/src/main/plugin/iso19139.ca.HNAP/layout/layout-custom-fields.xsl#L25

Any preference @joachimnielandt or @fxprunayre? I can do a pull request with these changes.

[josegar74]

I just notice you have a similar setup in this schema:

https://github.com/metadata101/dcat-ap/blob/main/src/main/java/org/fao/geonet/schema/dcatap/util/XslUtil.java

That is used in

dcat-ap/src/main/plugin/dcat-ap/layout/utility-fn.xsl

Line 29 in 9d9d5bc

xmlns:java2="java:org.fao.geonet.schema.dcatap.util.XslUtil"

I'll do a pull request to move all these other methods to this class and update the related xsl files

[josegar74]

@fxprunayre / @joachimnielandt do you know why is required the use of harvesterSetting.findOneByNameAndStoredValueLike (new method), to query by uuid using LIKE, while exists a method harvesterSetting.findOneByNameAndStoredValue?

dcat-ap/core-geonetwork-patches/patch-core-for-dcat-plugin.patch

Line 143 in 9d9d5bc

+ HarvesterSetting uuidSetting = harvesterSetting.findOneByNameAndStoredValueLike("uuid", metadata.getHarvestInfo().getUuid());

We can add it to core GeoNetwork the method if required, but would be good to have clear why is required, or otherwise update the code to use harvesterSetting.findOneByNameAndStoredValue.

---

Another question, I see most of the XslUtil methods are used in the patch for iso19139 indexing. I don't see why would be required to update the iso19139 (or other ISO schema indexing). Can you explain the reason for this?

[joachimnielandt]

@fxprunayre / @joachimnielandt do you know why is required the use of harvesterSetting.findOneByNameAndStoredValueLike (new method), to query by uuid using LIKE, while exists a method harvesterSetting.findOneByNameAndStoredValue?

This was introduced by @CMath04 but I cannot recall the exact reason for this myself (Informatievlaanderen/metadata-geonetwork@a41f094). It had to do with "resource URI pattern configuration for CSW and geonet harvesters", I'm assuming this was to support wildcards in a specific harvester setting? I cannot see a reference to this setting in our current environment though. Could you elaborate, @CMath04 ?

edit: Figured out why this setting was elusive... there is a frontend bug in our fork which doesn't render this specific setting.
It should display this:

An example setting would then be: resourceUriPattern:'https://www.dov.vlaanderen.be/geonetwork/srv/resources/{resourceType}/{resourceUuid}'. The harvestersetting uuid is still a regular value though, so it's not clear to me why it would need a LIKE based on the sample configuration.

edit2: could the reason be case sensitivity in our old system @CMath04 ? like on a mysql server behaves differently than on a postgresql server iirc

[CMath04]

@josegar74 @fxprunayre @joachimnielandt
This resourceURIPattern was set in place to make sure that the @rdf:about of the various rdf:Resource from converted ISO19139 metadata would be correct.
As you know, DCAT-AP dcat:Dataset, dcat:DataService, and others resources must have a proper URI in the @rdf:about attribute. By default, for ISO19139, we generate this URI at conversion time by using the current GN domain name followed by /srv/resources/{records|datasets|services|distributions}/{<uuid>}.
However, for harvested records, some organizations (rightfully) asked for the domain to be their own. For that reason, we added, at the harvester level, the possibility to configure the URI pattern to use.

When an ISO19139 record is converted to DCAT, this value will be used. I believe this is correctly implemented in the micro-service at the moment.

Now, for the linking between related datasets and services in DCAT, the link is done via the related resource URI (not the record URI but the dataset or the service URI).
This means this value needs to be indexed and searchable for the GetRelated API to work.
This value was added in a dedicated rdfResourceIdentifier index field in both DCAT-AP metadata and ISO19139. For ISO19139, this value is compiled at index time. Doing this enable the link between DCAT and ISO19139 record to be made.

Concerning the method findOneByNameAndStoredValueLike, I can’t tell for sure why this was added given that the method findOneByNameAndStoredValue already exists. This was indeed likely related to the difference between SQLServer and Postgres.

[CMath04]

@josegar74, I fixed the missing Resource URI pattern text box from the CSW harvester setting (See commit Informatievlaanderen/metadata-geonetwork@57c3337).
Do you want me to update the patch file or is the commit enough ?