inference-manager-engine/inference-manager-engine

The inference-manager-engine manages inference runtimes (e.g., vLLM and Ollama) in containers, load models, and process requests.