feat(metrics): calculate remaining capacity in pg connection pool

[iainsproat]

Description & motivation

The knex metric for 'free' connections only reports the number of existing created connections which are free prior to being garbage collected.

It does not take into account the additional capacity for creating new connections up to the maximum assigned to the connection pool. Additionally, these created connections can be removed as part of a garbage collection (aka 'reaping') process. As such the existing metric for free connections is unreliable. See #2441 for deeper investigation of the issue.

This PR adds a new metric which calculates the remaining capacity for adding additional new connections, accounting for the demand from existing in-use connections, pending acquisitions of these existing connections, and pending creations of new connections.

This PR also add a new metric for the pending creations of new connections.

Changes:

• new metric speckle_server_knex_remaining_capacity
• new metric speckle_server_knex_pending_creates

Adds the above to the server, webhook service, file import service, and the preview service.

To-do before merge:

Screenshots:

The new metrics are in yellow (remaining capacity), and red (pending creations). The previous metric is in blue and is not accurate (see timestamp at 19:03, for example).

We can see here in this close up that the remaining capacity, yellow, is at zero, despite there not being the maximum of used connections, green. This is because the remaining capacity takes into account pending acquisitions and creations. In this case it is pending creations, see in red which reduce the capacity to zero.

Validation of changes:

See PR #2441 for the validation of changes

Checklist:

• My pull request follows the guidelines in the Contributing guide?
• My pull request does not duplicate any other open Pull Requests for the same update/change?
• My commits are related to the pull request and do not amend unrelated code or documentation.
• My code follows a similar style to existing code.
• I have added appropriate tests.
• I have updated or added relevant documentation.

References

[iainsproat]

I have made a contribution to the upstream dependency to enable it to provide these metrics in a more concise manner. Vincit/tarn.js#80