Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons
All categories

LLM Infrastructure

Tools for serving, routing, evaluating and monitoring large language models

206tools listed

Pricing mix

  • 102 Free
  • 85 Freemium
  • 19 Paid

121 tools match

  • Vicuna

    www.lmsys.org

    A LLaMA-based research chatbot trained on user-shared conversations to handle multi-turn dialogue.

    Free
  • SmartBird

    www.smartbird.ai

    Dedicated, fully managed AI infrastructure designed around your workloads

    Paid
  • VModel

    www.vmodel.ai

    Run pre-trained AI models and deploy custom models through a unified REST API.

    Paid
  • Local AI Playground

    www.localai.app

    Manage and verify AI models, run them offline on a CPU, and serve responses locally.

    Free
  • Granica

    www.granica.ai

    Enterprise AI infrastructure for data cost reduction, tabular models and agent state

    Paid
  • Prodia

    www.prodia.com

    Inference API for generative video and image models

    Paid
  • Neverinstall

    www.neverinstall.com

    Private cloud, virtual desktops and GPU infrastructure on owned servers or cloud environments.

    Paid
  • Cerebrium

    www.cerebrium.ai

    Serverless GPU infrastructure for deploying and scaling voice agents, LLMs, video models, and custom AI workloads.

    Paid
  • Prediction Guard

    www.predictionguard.com

    Self-hosted AI agent control plane with runtime policies, compatible APIs, and enterprise governance.

    Paid
  • GGML

    www.ggml.ai

    A cross-platform tensor library for running large machine learning models on commodity hardware.

    Free
  • Gold Retriever

    www.jina.ai

    Create retrieval plugins that let ChatGPT search and use your own text data.

    Free
  • ZBrain

    www.zbrain.ai

    Enterprise agentic AI governance with runtime policy enforcement, agent development and audit trails.

    Paid
  • Amazon Bedrock

    aws.amazon.com

    Amazon Bedrock provides foundation models, customization tools, and agent infrastructure for production AI applications.

    Paid
  • xTuring

    xturing.stochastic.ai

    Open-source LLM customization with LoRA fine-tuning and perplexity evaluation examples.

    Free
  • Inworld AI

    www.inworld.ai

    Speech APIs and open-model inference for developers building consumer AI applications.

    Paid
  • EnergeticAI

    www.energeticai.org

    Pre-trained English embeddings and text classifiers for Node.js, with a focus on serverless cold-start performance.

    Free
  • AI SDK

    sdk.vercel.ai

    An open-source TypeScript toolkit for building AI applications and agents with streaming and multi-provider support.

    Free
  • Avian

    www.avian.io

    OpenAI-compatible AI inference and dedicated NVIDIA GPU compute for developers.

    Paid
  • magpie

    www.usemagpie.ai

    A local gateway for choosing models, sharing subscriptions and masking secrets across coding agents.

    Free
  • BAML

    www.boundaryml.com

    An open-source programming language for AI agents with typed outputs, concurrency and built-in observability.

    Free
  • lithos-metal

    www.github.com

    Open-source LLM inference and local serving for Apple silicon Macs using Metal megakernels and DSpark decoding.

    Free
  • openTPU

    www.github.com

    Open-source AI accelerator with hardware design, a compiler, a simulator and profiling tools in one repository.

    Free
  • AxisRL

    www.github.com

    Open-source agentic reinforcement learning post-training with SGLang rollout and Megatron training.

    Free
  • LLM Graph Builder

    www.github.com

    Build Neo4j knowledge graphs from documents, videos and web pages, then query the data through chat.

    Free
  • KaaS

    www.github.com

    Compile notes, documents, and transcripts into a self-hosted Markdown wiki with cited answers and MCP access.

    Free
  • tokenizers

    www.github.com

    Open-source tokenization library for encoding and decoding text in research and production.

    Free
  • mistral.rs

    www.github.com

    Open-source LLM inference with multimodal models, local coding agents, compatible APIs, and Python and Rust SDKs.

    Free
  • AxonHub

    www.github.com

    Open-source AI gateway for model routing, SDK compatibility, request tracing and cost control.

    Free
  • tare

    www.github.com

    Audit Claude Code token usage, diagnose limits, and trace context overhead from local session logs.

    Free
  • agent-sandbox

    www.github.com

    Open-source Kubernetes orchestration for stateful sandboxes, AI agent runtimes, and reinforcement learning workloads.

    Free
  • gepa

    www.github.com

    Open-source optimization of prompts, code, agent architectures, and configurations using LLM reflection.

    Free
  • Swiftlet

    www.github.com

    Run Qwen Mixture-of-Experts models locally on Macs and iPhones by streaming expert weights from storage.

    Free
  • AI-Infra-Guard

    www.github.com

    Open-source AI red teaming with agent, skill, MCP, infrastructure and LLM security scanners.

    Free
  • Antigravity-Manager

    www.github.com

    Desktop account management, API protocol conversion, and request routing for AI services.

    Free
  • Automodel

    www.github.com

    Open-source PyTorch distributed training and fine-tuning for language and vision-language models

    Free
  • TokenHub

    www.github.com

    Open-source AI gateway for enterprise model routing, access control, usage attribution, and cost governance.

    Free