[DEV-9353] Support $optionalRef syntax JsonFileSourceProcessor

[facetoe]

Add support for $optionalRef, inspired by JSONSchema ref. It is $optionalRef and not $ref as the JSONSchema spec requires that the referenced file exist or the parser should fail, so invented this version to keep the semantics clear. We also only support file:// refs.

The basic idea is that you can add a reference like so:

{
  "metricsDatabase": {
    "$optionalRef": "file:///home/coe/worktmp/test.json"
  }
}

And provided the destination path exists, the entire entry (including $optionalRef key) will be replaced with what is in the destination. In the event that the destination is absent, the entire containing object is removed (ie, no metricsDatabase key in structure anymore).

The idea here is that we can optionally mount things in Kubernets, such as DB secrets, config etc, and then reference it using this feature. In the event it exists, the application is configured to use the config. If it is absent, the config is removed. The result is a type of dynamic configuration based on what has been injected into the environment.

Description

See commit message.

Associated tasks

NA

Test Steps

Wrote tests

Checklist

Code

• [ x] I have performed a self review of my own code (including checking issues raised when creating the PR).
• [ x] I have added/updated unit tests for these changes, and if not I have explained why they are not necessary.
• [ x] I have commented my code in any hard-to-understand or hacky areas.
• [ x] I have handled all new warnings generated by the compiler or IDE.
• [ x] I have rebased onto the target branch (usually main).

Security

When developing applications, use following guidelines for information security considerations:

• Access to applications should be protected with security keys/tokens or usernames and passwords;
• All sessions are encrypted if possible;
• All application input is sanitised before being acted on (ie SQL statements, etc);
• Log messages, and especially client-facing ones, must be handled securely and must not leak credentials information (internal URLs, passwords, tokens).

• [ x] I have considered if this change impacts information security and made sure those impacts are handled.

Documentation

• I have updated the changelog.
• I have updated any documentation required for these changes.

Breaking Changes

• I have considered if this is a breaking change and will communicate it with other team members by posting it on the Slack breaking-changes channel.

Please leave a summary of the breaking changes here and then post it on the Slack breaking-changes channel to notify the team about it.

[RabidGhost]

Overall looks good. Just some changes to test clarity, and to resolve as many issues as possible before reporting problems.

[RabidGhost]

Tests look good! Some optional things to consider, but we could merge as is if you are happy