How to use /backend/*
endpoints
#1628
Unanswered
TheDarkTrumpet
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Good morning,
This morning I was about to open a bug request surrounding the ability to shutdown a model from the API. The use case is if I want to test different models, on a multi-GPU machine, to be able to shutdown the currently running one and load the new one in the case I need it.
I came across a pull request, #938 which mentioned a endpoint called
/backend/shutdown
. Upon playing with that (which appears to be a post), I'm unclear on what I should be sending to it.I also saw one for
/backend/monitor
, which I figured would be simpler. It's a GET request, and figured it'd show me the general status. Upon executing the request below, I get a 422 error code with an incredibly unhelpful error message. I was looking for API docs (like Text-Gen-WebUi has), as an endpoint, that can describe the APIs and what's expected and nada.Results in:
I'd love to debug this further on my end. I already dug into the source code, but my go skills aren't quite up to snuff. https://github.com/mudler/LocalAI/blob/master/api/api.go has some of the code, and https://github.com/mudler/LocalAI/blob/master/api/localai/backend_monitor.go has the rest for the monitor.
Any help or pointers would be useful.
Thanks,
Beta Was this translation helpful? Give feedback.
All reactions