Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons
All categories

LLM Infrastructure

Tools for serving, routing, evaluating and monitoring large language models

206tools listed

Pricing mix

  • 102 Free
  • 85 Freemium
  • 19 Paid

206 tools

Filters

  • SigNoz

    www.github.com

    OpenTelemetry-native observability for logs, metrics, traces, infrastructure, and AI applications.

    Freemium
  • plano

    www.github.com

    Open-source AI proxy for agent orchestration, LLM routing, guardrails, and OpenTelemetry observability.

    Free
  • Heretic

    www.github.com

    Open-source language model modification for refusal suppression, shorter responses, and reduced sycophancy.

    Free
  • slime

    www.github.com

    Open-source LLM reinforcement learning framework connecting Megatron training with SGLang rollout.

    Free
  • marin

    www.github.com

    Open-source framework for foundation model training, data preparation, evaluation, and reproducible experiments.

    Free
  • LMCache

    www.github.com

    Open-source KV cache management for reusable, persistent caching across LLM inference engines and storage backends.

    Free
  • CocoIndex

    www.github.com

    Open-source incremental data engine that keeps context fresh for AI agents and LLM applications.

    Free
  • vLLM

    www.github.com

    Open-source LLM inference and serving with PagedAttention, distributed execution and an OpenAI-compatible API.

    Free
  • Rig

    www.github.com

    Rust library and plugin-based terminal coding agent for building LLM applications.

    Free
  • NeMo

    www.github.com

    Open-source framework for creating, customizing, and deploying speech recognition, text-to-speech, and speech LLMs.

    Free
  • context-mode

    www.github.com

    A free local plugin that sandboxes coding-agent tool output and retrieves relevant results through SQLite search.

    Free
  • Olostep Web Data API

    www.olostep.com

    Web scraping, crawling, search and monitoring API for engineering teams building AI applications.

    Freemium
  • ZenMux

    www.zenmux.ai

    Unified AI model gateway with API access, automatic routing, usage analytics, and output compensation.

    Freemium
  • ChatBetter

    www.chatbetter.com

    Multi-model AI research focused on task routing, source-grounded responses, and automated output evaluation.

    Freemium
  • Privatemode AI

    www.privatemode.ai

    Confidential AI chat, coding, and API access with encryption that protects data during processing.

    Freemium
  • Scoopika

    www.scoopika.com

    Open-source toolkit for building multimodal AI agents and LLM-powered web applications.

    Freemium
  • AIProxy

    www.aiproxy.pro

    Secure AI API keys, monitor requests, and control usage in Swift apps.

    Freemium
  • Prem AI

    www.premai.io

    Prem AI provides private enterprise agents and encrypted GPU infrastructure for sovereign AI.

    Freemium
  • Align AI

    www.tryalign.ai

    Analytics for LLM chatbots, with conversation monitoring, cohort analysis and auto-generated action items.

    Freemium
  • Confident AI

    www.confident-ai.com

    LLM evaluation, production monitoring, and expert annotation in a shared AI quality workspace.

    Freemium
  • Datasaur

    www.datasaur.ai

    Private enterprise AI agents and custom models deployed inside your own infrastructure.

    Paid
  • Klu.ai

    www.klu.ai

    Evaluate AI models on real workflows and track when new releases improve performance or cost.

    Paid
  • LiteLLM

    www.github.com

    Open-source AI gateway and Python SDK for calling LLM providers through a unified OpenAI-compatible interface.

    Free
  • Get Odin AI

    www.getodin.ai

    Build and deploy AI agents that automate enterprise workflows across connected apps, data, and systems.

    Paid
  • Anthropic

    www.anthropic.com

    AI research and Claude models for summarization, browser use, agentic coding, and knowledge work.

    Freemium
  • Hugging Face

    www.huggingface.co

    Discover, host and collaborate on machine learning models, datasets and applications.

    Freemium
  • Langdock

    www.langdock.com

    Company-wide AI chat, agents, workflows and governance with access to multiple AI models.

    Paid
  • TrueFoundry AI Gateway

    www.truefoundry.com

    Unified model access, routing, monitoring, and governance for enterprise AI applications.

    Freemium
  • BloombergGPT

    www.gpt3demo.com

    BloombergGPT is a finance-focused large language model trained to support financial natural language processing tasks.

    Freemium
  • Babel Cloud

    www.babel.cloud

    An AI lab presenting personal assistance, investment research, and environments for agent development.

    Freemium
  • Modal

    www.modal.com

    Serverless cloud infrastructure for AI inference, model training, agent sandboxes, and compute-heavy applications.

    Freemium
  • CopilotKit Intelligence

    www.copilotkit.ai

    Build in-app AI agents with generative UI, shared state, automatic learning, and product analytics.

    Freemium
  • Dataloop

    www.dataloop.ai

    AI data infrastructure for unstructured data, multimodal pipelines, model deployment, and human feedback.

    Freemium
  • Layerbrain

    www.layerbrain.com

    Computer World Models designed to generate, perceive, and interact with working software entirely from pixels.

    Freemium
  • fal

    www.fal.ai

    Generative media APIs, serverless model deployment, and GPU compute for developers

    Paid
  • Unified AI development, training, and inference across a global network of GPU cloud providers.

    Freemium