Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

Magnitude

Open-source inference engine that tunes local models for your hardware and connects them to coding agents.

Magnitude is a free, open-source inference engine that tunes model kernels for local hardware, runs open models, and connects them to AI agents. Magnitude is for developers and agent users, with desktop and CLI access on macOS, Linux, and Windows under the Apache 2.0 license.

Updated 9 Nov 20266 sections

Magnitude website screenshot

What is Magnitude?

Magnitude is an open-source inference engine that runs open models and optimizes their kernels for the hardware available. Magnitude is designed for developers and people using AI agents who want models to run on their own machines. The project provides a desktop app and CLI, supports macOS, Linux, and Windows, and connects local models to existing agents rather than requiring users to replace their agent workflow.

Magnitude supports Apple Silicon, NVIDIA, AMD, and Intel GPUs, along with CPU-only operation. The source specifies no fixed hardware minimum: smaller machines run smaller models, while additional memory allows larger models. Magnitude keeps prompts, files, and models on the user's machine and does not require internet access after a model has been downloaded.

What Magnitude can do

Magnitude combines hardware-specific inference optimization with features for running agent workloads locally. Magnitude tunes each downloaded model, includes speculative decoding drafters, and manages memory across agent sessions. The project also provides direct connections to named agents and an API path for other clients, allowing local inference to fit into existing development workflows.

  • Magnitude automatically tunes kernels for each model on the user's hardware.
  • Magnitude includes a speculative decoding drafter with each model.
  • Magnitude uses quantized KV memory that is freed when agents stop.
  • Magnitude shares prefix caches across concurrent sessions.
  • Magnitude connects to Pi, OpenCode, Hermes, Codex, Claude Code, and Cline.
  • Magnitude supports other clients through an OpenAI-compatible API.

How Magnitude works

Magnitude can be installed through terminal commands or downloaded installers, with both desktop and CLI interfaces available. When a user downloads a model, Magnitude tests different ways to divide computation on the actual chip and retains the fastest combination that does not change the model's output. The source says this tuning can take up to a minute.

Magnitude offers agent connections through a single-click workflow and supports additional clients through an OpenAI-compatible API. Magnitude also provides documentation for running as a remote service accessible to other computers and for running in Docker. The supported model list is published at magnitude.dev/models, with optimized kernels focused on popular open-weight families.

Who Magnitude is for

Magnitude serves developers and AI agent users who want to run open models on their own hardware. Magnitude supports workflows involving existing coding agents, offline model use, and concurrent sessions. Users can choose local desktop or CLI operation, or consult the remote-server documentation when other computers need access to the inference service.

  • Magnitude can supply local models to existing coding agents.
  • Magnitude can run downloaded models without an internet connection.
  • Magnitude can support concurrent agent sessions with shared prefix caches.
  • Magnitude can run as a service that other computers use.

Magnitude pricing and access

Magnitude is free and open source under the Apache 2.0 license, and the project states that there are no token costs. Magnitude is available through the public repository, terminal installation commands, and downloadable installers. Installer options cover macOS on Apple silicon and Intel, Windows x64, and Linux packages for x64 and ARM64.

Magnitude strengths and limitations

Magnitude combines local data handling, hardware-specific tuning, and connections to existing agents. Magnitude also supports multiple operating systems and both GPU and CPU execution. These capabilities make the project relevant to local inference workflows, but model choice and model size still depend on the supported model families and the memory available on the user's machine.

  • Magnitude requires a model download before offline use.
  • Magnitude adds hardware-tuning time when a model is downloaded.
  • Magnitude runs larger models only when sufficient memory is available.
  • Magnitude focuses optimized kernels on popular open-weight model families.

Magnitude FAQ

What is Magnitude?

Magnitude is an open-source inference engine that runs open models and automatically tunes their kernels for local hardware. Magnitude includes a desktop app and CLI and connects local models to existing AI agents through direct connections or an OpenAI-compatible API.

Is Magnitude free?

Magnitude is free and open source under the Apache 2.0 license, with no token costs. Magnitude is available through terminal installation commands and downloadable installers, with desktop and CLI interfaces for supported operating systems.

What hardware and operating systems does Magnitude support?

Magnitude supports macOS, Linux, and Windows on Apple Silicon, NVIDIA, AMD, or Intel GPUs, as well as CPU-only machines. Magnitude has no fixed hardware minimum, but smaller machines run smaller models and more memory enables larger models.

Which agents work with Magnitude?

Magnitude offers single-click connections to Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Magnitude also provides an OpenAI-compatible API so other clients can connect to models running through the inference engine.

Can Magnitude run offline?

Magnitude can run without internet access once a model has been downloaded. Magnitude keeps prompts, files, and models on the user's machine, making local model execution available without sending those materials to an external inference service.

How does Magnitude optimize model inference?

Magnitude automatically tunes model kernels on the user's actual hardware during model download. Magnitude tests different ways to split computation and keeps the fastest combination that does not change the model's output, with tuning taking up to a minute.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like Magnitude

lithos-metal

www.github.com

Open-source LLM inference and local serving for Apple silicon Macs using Metal megakernels and DSpark decoding.

Free

TurboFieldfare

www.github.com

Run Gemma 4 26B-A4B locally on Apple Silicon Macs using about 2 GB of RAM.

Free

llama-swap

www.github.com

On-demand model switching for local AI servers with OpenAI and Anthropic API compatibility.

Free

llama.cpp

www.github.com

Open-source LLM and vision-language model inference in C/C++ for local and cloud hardware.

Free