LLM Infrastructure
Tools for serving, routing, evaluating and monitoring large language models
206tools listed
Pricing mix
- 102 Free
- 85 Freemium
- 19 Paid
Open-source model routing for AI agents, with Rust embedding and gateway integrations.
FreeMemory-aware GGUF inference with CPU and GPU offloading and an OpenAI-compatible server.
FreeOpen-source C99 inference engine for running Kimi K3 on a CPU without BLAS, a framework or a GPU.
FreeAn open-source differential privacy gateway for AI agents querying sensitive data over MCP.
FreeOpen-source infrastructure for running, snapshotting and forking agent environments across machines.
FreeOpen-source Rust machine learning framework for model training, GPU inference, and browser-based models.
FreeOpen-source LLM data preparation with reusable operators, visual pipelines, and data generation and cleaning workflows.
FreeConnect ChatGPT accounts to apps through an OpenAI-compatible proxy, TypeScript SDK and browser sign-in.
FreeOpen-source, reasoning-based document retrieval using hierarchical tree indexes instead of vector databases.
FreeOpen-source AI proxy for agent orchestration, LLM routing, guardrails, and OpenTelemetry observability.
FreeOpen-source language model modification for refusal suppression, shorter responses, and reduced sycophancy.
FreeOpen-source LLM reinforcement learning framework connecting Megatron training with SGLang rollout.
FreeOpen-source framework for foundation model training, data preparation, evaluation, and reproducible experiments.
FreeOpen-source KV cache management for reusable, persistent caching across LLM inference engines and storage backends.
FreeOpen-source incremental data engine that keeps context fresh for AI agents and LLM applications.
FreeOpen-source LLM inference and serving with PagedAttention, distributed execution and an OpenAI-compatible API.
FreeOpen-source framework for creating, customizing, and deploying speech recognition, text-to-speech, and speech LLMs.
FreeA free local plugin that sandboxes coding-agent tool output and retrieves relevant results through SQLite search.
FreeOpen-source AI gateway and Python SDK for calling LLM providers through a unified OpenAI-compatible interface.
FreeCompare LLM pricing and benchmarks, browse AI datasets and agent tools, and find AI jobs.
FreeA free prompt engineering playground for composing prompts, testing outputs, and estimating inference costs.
FreeOpen-source AI API gateway for unified subscription access and shared quota distribution.
FreeOpen-source vector database for storage-efficient semantic search and local RAG on personal data.
FreeOpen-source knowledge framework for document search, RAG, reasoning agents and a self-maintaining wiki.
FreeOfficial Claude API command-line tool for messages, agents, sessions, file uploads, and API scripting.
FreeCompress AI coding agent output and route heavy tasks to cheaper models with shareable pipelines.
FreeConnect coding agents to free, paid, subscription, and local models through a searchable model catalog.
FreeA local model gateway for routing coding agents across providers, models, accounts, and tools.
Free





























