Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

reef

Open-source infrastructure for improving agent model weights, prompts, rules, and skills from feedback.

reef is an open-source infrastructure project that connects agent inference, feedback, learning, and versioned delivery. reef helps developers and researchers train model weights or improve agent prompts, rules, and skills. reef is available through PyPI or a source checkout under the Apache-2.0 license.

Updated 9 Nov 20266 sections

reef website screenshot

What is reef?

reef is an open-source infrastructure project for continually self-improving agents. reef connects serving, feedback collection, learning, and versioned delivery in a shared workflow. Developers can use reef to train model weights with Slime and SGLang or improve the agent harness, including prompts, rules, and skills.

reef distinguishes between model weight training, harness optimization, and test-time training. These paths serve different goals: adapting a trainable model, improving an agent's surrounding instructions and tools, or repeatedly working on a problem with a measurable objective. The deployment's recipe determines which learning surface its scenarios update.

What reef can do

reef combines live agent serving with mechanisms for turning interactions into evaluated updates. Feedback can include numeric scores, text, or structured objects, depending on the recipe. Model updates and harness edits follow versioned publication workflows, allowing subsequent requests or connected clients to use accepted changes.

  • reef serves inference requests through OpenAI- and Anthropic-compatible endpoints.
  • reef records interactions and matches feedback to the relevant records.
  • reef trains model weights through configured training recipes.
  • reef evolves agent prompts, rules, and skills as a tree of text files.
  • reef evaluates candidate updates before publishing accepted versions.
  • reef loads accepted model checkpoints without restarting the service.

How reef works

reef organizes learning into serving, observing, growing, and committing. Agent requests create interaction records, and later reports identify those interactions using receipt headers. Eligible records feed the configured recipe, which produces an update. Evaluation and selection policies then determine whether a candidate becomes a published artifact.

reef accepts provider-compatible inference requests and uses a scenario header to associate traffic with a scenario. A new scenario name uses the deployment's configured recipe rather than selecting a different recipe. Harness deployments evaluate candidate file trees with the real agent on tasks and publish winning snapshots for clients to retrieve.

Who reef is for

reef serves developers and researchers who want agents to improve from feedback rather than remain fixed after deployment. Weight-training users need a trainable model, supported GPU infrastructure, and usable feedback. Harness users need a model endpoint, representative tasks, and an evaluator, but do not need local training GPUs.

  • reef supports coding agent skill and rule improvement from scored feedback.
  • reef supports learning from demonstrations or imported examples through SFT.
  • reef supports offline tasks with reference answers through task-specific recipes.
  • reef supports online learning from user sessions.
  • reef supports discovery workloads with correctness checks and measurable objectives.

reef pricing, license, and installation

reef is distributed as an open-source project under the Apache-2.0 license. The repository provides installation through the reef-infra package on PyPI or through a source checkout. The documentation recommends uv for package management and directs development and training-example users to use the source checkout.

reef requires the git-lfs system package for artifact and checkpoint functionality and initializes Git LFS locally for artifact repositories. A minimal deployment can run as a pure inference server. The harness tutorial additionally requires an OpenAI-compatible endpoint, Node.js, and npm for the pi agent, with provider settings supplied through configuration.

reef strengths and limitations

reef brings inference, feedback handling, evaluation, and release history into one learning workflow. Support for both weights and harnesses gives developers distinct adaptation paths, including a harness path without local training GPUs. Operational requirements still depend on the chosen recipe, training components, model endpoint, and execution environment.

  • reef weight-training deployments require a supported GPU training stack.
  • reef warns that some recipes and examples can fail during the control plane and workers refactor.
  • reef ships SFT with reef-infra, while other recipe implementations remain in the repository cookbook.
  • reef advises storing published harness trees outside the agent's working project to prevent sessions from changing future harness files.

reef FAQ

What is reef?

reef is an open-source infrastructure project for continually self-improving agents. reef connects inference, feedback, learning, and versioned delivery, supporting both model weight training and changes to an agent's prompts, rules, and skills.

Is reef open source?

reef is open source under the Apache-2.0 license. reef is available as the reef-infra package on PyPI or through its GitHub source repository, with the source checkout recommended for development and training examples.

Does reef require GPUs?

reef requires a supported GPU training stack for model weight training, but harness optimization does not require local training GPUs. reef harness optimization instead needs a model endpoint, representative tasks, and an evaluator.

Which APIs does reef support?

reef provides OpenAI- and Anthropic-compatible inference endpoints, including /v1/chat/completions, /v1/responses, and /v1/messages. reef accepts the provider's request body and adds an interaction receipt header that later feedback reports can reference.

How does reef improve an agent's harness?

reef stores an agent harness as a tree of text files and edits candidate copies using step traces. reef evaluates candidates by running the real agent on tasks, then publishes a winning candidate as a snapshot that clients can retrieve.

Can reef update a model without restarting?

reef loads a training checkpoint as the new inference snapshot after the checkpoint passes evaluation. reef then serves later inference requests using the current version without restarting, connecting feedback-driven training to live model delivery.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like reef

SGLang

www.github.com

Open-source inference and serving framework for large language, vision-language, and diffusion models.

Free

slime

www.github.com

Open-source LLM reinforcement learning framework connecting Megatron training with SGLang rollout.

Free

Modal

www.modal.com

Serverless cloud infrastructure for AI inference, model training, agent sandboxes, and compute-heavy applications.

Freemium

AxisRL

www.github.com

Open-source agentic reinforcement learning post-training with SGLang rollout and Megatron training.

Free