Docker model runner
Sometime in April this year, Docker added new feature called Docker Model Runner. It’s meant to streamline the process of pulling, running, and serving large language models (LLMs) and other AI models directly from Docker Hub or OCI-compliant registries. It integrates with Docker Desktop and Docker Engine, allows you to serve models via OpenAI-compatible APIs, package GGUF files as OCI, and interact with models from the command line. Features Pull and push models Serve models on OpenAI-compatible APIs Package and publish GGUF files as OCI Run AI models directly from the command line Manage local models and display logs Requirements Docker Model Runner is supported on the following platforms: ...