Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons
All categories

LLM Infrastructure

Tools for serving, routing, evaluating and monitoring large language models

206tools listed

Pricing mix

  • 102 Free
  • 85 Freemium
  • 19 Paid

187 tools match

  • Geschäft

    www.ora.ai

    The supplied URL returns a page-not-found notice and directs AI agents to /llms.txt.

    Freemium
  • AI SDK

    sdk.vercel.ai

    An open-source TypeScript toolkit for building AI applications and agents with streaming and multi-provider support.

    Free
  • Merge

    www.merge.dev

    Connect AI agents to business data, authenticated tools, and language models.

    Freemium
  • FlowiseAI

    www.flowiseai.com

    Open-source visual builder for AI agents, chatbots, and multi-agent workflows.

    Freemium
  • BenchLLM

    www.benchllm.com

    Evaluate LLM-powered apps with test suites, multiple evaluation strategies, a CLI, and quality reports.

    Freemium
  • Azure AI

    azure.microsoft.com

    Build, deploy, and govern AI applications and agents with models, databases, developer tools, and security services.

    Freemium
  • Backblaze

    www.backblaze.com

    S3-compatible cloud object storage for AI workloads, applications, archives, and secure computer backup.

    Freemium
  • Allen Institute for AI

    www.allenai.org

    Open AI models, scientific discovery tools, environmental modeling, and embodied AI research.

    Freemium
  • Aizzy

    www.aizzy.ai

    A privacy-first web interface for routing paid AI models, browsing in Ghost Mode, and keeping chats on-device.

    Freemium
  • AIToolBox.org

    www.aitoolbox.org

    AI tools for business image and text generation, customization, and hosting.

    Freemium
  • magpie

    www.usemagpie.ai

    A local gateway for choosing models, sharing subscriptions and masking secrets across coding agents.

    Free
  • Coasty

    www.coasty.ai

    Computer-use environments, agent trajectories and outcome evaluation for frontier-model research.

    Freemium
  • BAML

    www.boundaryml.com

    An open-source programming language for AI agents with typed outputs, concurrency and built-in observability.

    Free
  • ZeroShot

    www.tryzeroshot.com

    Coding-agent session monitoring with reusable skills, session handoffs, and convention-based PR reviews.

    Freemium
  • Open WebUI

    www.openwebui.com

    A self-hosted AI interface for local and cloud models, Python extensions, and community tools.

    Freemium
  • Muapi

    www.muapi.ai

    Generate AI images and videos in a web studio, or access generative-media models through a unified API.

    Freemium
  • Langtail

    www.langtale.ai

    Collaborative prompt management, testing, deployment, and AI security for product teams.

    Freemium
  • lithos-metal

    www.github.com

    Open-source LLM inference and local serving for Apple silicon Macs using Metal megakernels and DSpark decoding.

    Free
  • openTPU

    www.github.com

    Open-source AI accelerator with hardware design, a compiler, a simulator and profiling tools in one repository.

    Free
  • AxisRL

    www.github.com

    Open-source agentic reinforcement learning post-training with SGLang rollout and Megatron training.

    Free
  • LLM Graph Builder

    www.github.com

    Build Neo4j knowledge graphs from documents, videos and web pages, then query the data through chat.

    Free
  • KaaS

    www.github.com

    Compile notes, documents, and transcripts into a self-hosted Markdown wiki with cited answers and MCP access.

    Free
  • tokenizers

    www.github.com

    Open-source tokenization library for encoding and decoding text in research and production.

    Free
  • mistral.rs

    www.github.com

    Open-source LLM inference with multimodal models, local coding agents, compatible APIs, and Python and Rust SDKs.

    Free
  • AxonHub

    www.github.com

    Open-source AI gateway for model routing, SDK compatibility, request tracing and cost control.

    Free
  • Hillock

    www.github.com

    Local neuro-symbolic memory for offline AI, with verified fact retrieval and an OpenAI-compatible API.

    Freemium
  • tare

    www.github.com

    Audit Claude Code token usage, diagnose limits, and trace context overhead from local session logs.

    Free
  • agent-sandbox

    www.github.com

    Open-source Kubernetes orchestration for stateful sandboxes, AI agent runtimes, and reinforcement learning workloads.

    Free
  • gepa

    www.github.com

    Open-source optimization of prompts, code, agent architectures, and configurations using LLM reflection.

    Free
  • Swiftlet

    www.github.com

    Run Qwen Mixture-of-Experts models locally on Macs and iPhones by streaming expert weights from storage.

    Free
  • AI-Infra-Guard

    www.github.com

    Open-source AI red teaming with agent, skill, MCP, infrastructure and LLM security scanners.

    Free
  • Antigravity-Manager

    www.github.com

    Desktop account management, API protocol conversion, and request routing for AI services.

    Free
  • Automodel

    www.github.com

    Open-source PyTorch distributed training and fine-tuning for language and vision-language models

    Free
  • TokenHub

    www.github.com

    Open-source AI gateway for enterprise model routing, access control, usage attribution, and cost governance.

    Free
  • Repomix

    www.github.com

    Pack repositories into AI-friendly files with token counts, configurable exclusions, and optional code compression.

    Free
  • semantica

    www.github.com

    Open-source context graphs, knowledge graphs, deterministic reasoning, and decision provenance for AI systems.

    Free