Any metrics for Gliner server?

[vrdn-23]

Thanks for adding a serving component @Ingvarstep

One thing we're looking to evaluate for production readiness is metrics for the server, like number of requests running/waiting in the queue etc, for scaling decisions. Is this something that is supported but not documented or is it something that would we like to help implement?

I'd be happy to help contribute if needed!

[Ingvarstep]

Hi @vrdn-23, this is handled by Ray Serve, which we use under the hood. It already emits Prometheus metrics for queued/ongoing/processing queries per deployment and replica, plus batch-size histograms from @serve.batch. So it's supported but not documented on our side. Contributions very welcome, a docs pass covering how to scrape those metrics (and ideally exposing the metrics port in our Dockerfile and CLI) would be a great starting point. GLiNER-specific custom metrics on top of that would be a nice follow-up.