Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

GPT Prompt Engineer

Open-source prompt generation, testing and ELO ranking in Google Colab or local Jupyter notebooks.

GPT Prompt Engineer is an open-source prompt engineering tool that generates, tests and ranks prompts for defined tasks. GPT Prompt Engineer serves developers and AI builders comparing prompt performance through notebooks. Access is through an MIT-licensed GitHub repository, with provider API keys required and potential model usage costs.

Updated 9 Nov 20266 sections

At a glance

Pricing
FreeMIT-licensed open-source project. Users supply an OpenAI or Anthropic API key; the README warns that generating many prompts can become expensive.
Added
9 Nov 2026
GPT Prompt Engineer website screenshot

What is GPT Prompt Engineer?

GPT Prompt Engineer is an open-source notebook project for generating and evaluating prompts against a defined task. GPT Prompt Engineer takes a use-case description and test cases, creates candidate prompts, and compares their performance. The repository presents prompt engineering as experimentation and provides a workflow for making those comparisons through testing and rankings.

GPT Prompt Engineer is available through a public GitHub repository with an MIT license. The project includes a general prompt engineering notebook, a classification version, a Claude version and conversion notebooks. The documented examples cover tasks such as landing page headlines and personalized email responses.

What GPT Prompt Engineer can do

GPT Prompt Engineer combines candidate generation with evaluation against test cases. The general workflow ranks competing prompts using ELO ratings, while the classification notebook checks outputs against expected labels. The Claude version adds automatic test-case generation and multiple input variables, and optional integrations record experiment details or trace prompt chains.

  • GPT Prompt Engineer generates candidate prompts using GPT-4, GPT-3.5-Turbo or Claude 3 Opus.
  • GPT Prompt Engineer tests prompts and presents ELO rankings in a sorted table.
  • GPT Prompt Engineer evaluates classification outputs against expected true or false labels.
  • GPT Prompt Engineer automatically generates test cases in the Claude version.
  • GPT Prompt Engineer logs configurations, prompts, test cases and rankings to Weights & Biases when enabled.
  • GPT Prompt Engineer traces prompt chains and responses through Portkey when enabled.

How GPT Prompt Engineer works

GPT Prompt Engineer runs in Google Colab or a local Jupyter notebook. Users open the appropriate notebook and add an OpenAI API key for the OpenAI workflow or an Anthropic API key for the Claude version. The GPT workflow requires a task description and test cases before candidate generation begins.

GPT Prompt Engineer lets users choose how many candidate prompts to generate, then calls generate_optimal_prompt to generate, test and rate them. The Claude workflow instead accepts input variables and generates test cases from those variables and the task description. Results appear as a descending ELO table; the classification version displays prompt scores.

Who GPT Prompt Engineer is for

GPT Prompt Engineer suits developers and AI builders who can configure notebooks and want to compare prompts for a specific task. The source provides examples spanning content generation, personalized responses and classification. These examples describe prompt evaluation workflows rather than a separate application for publishing content or managing email.

  • GPT Prompt Engineer can compare prompts for generating landing page headlines.
  • GPT Prompt Engineer can evaluate personalized email response prompts using named input variables.
  • GPT Prompt Engineer can score classification prompts against expected true or false outputs.

GPT Prompt Engineer pricing and access

GPT Prompt Engineer is an MIT-licensed open-source project obtained through GitHub, rather than a documented subscription service. Running the notebooks requires the relevant provider API key. The README explicitly warns that generating many candidate prompts can become expensive, so access to the source code should be distinguished from model usage costs.

GPT Prompt Engineer strengths and limitations

GPT Prompt Engineer brings prompt generation, test-case evaluation and comparative ranking into a documented notebook workflow. Multiple notebook variants address general generation, classification and Claude-based experimentation. Optional Weights & Biases logging and Portkey tracing provide ways to inspect configurations, results and prompt chains alongside the final rankings.

  • GPT Prompt Engineer requires users to configure a notebook and supply a provider API key.
  • GPT Prompt Engineer may incur substantial model usage costs when many prompts are generated.
  • GPT Prompt Engineer's documented classification version uses true or false labels; broader class support is listed as a contribution idea.

GPT Prompt Engineer FAQ

What is GPT Prompt Engineer?

GPT Prompt Engineer is an open-source project that generates, tests and ranks prompts from a task description and test cases. GPT Prompt Engineer uses notebook workflows and ELO ratings to compare candidate prompts, with separate versions for classification and Claude-based experimentation.

Is GPT Prompt Engineer free?

GPT Prompt Engineer is available as MIT-licensed open-source code on GitHub. Running GPT Prompt Engineer requires an OpenAI or Anthropic API key, depending on the notebook, and the README warns that generating many prompts can become expensive.

How do you run GPT Prompt Engineer?

GPT Prompt Engineer runs in Google Colab or a local Jupyter notebook. Users open the relevant notebook, add the required provider API key, define the task and test inputs, and run the generation and evaluation workflow to obtain ranked prompts.

Which models does GPT Prompt Engineer support?

GPT Prompt Engineer documents prompt generation with GPT-4, GPT-3.5-Turbo and Claude 3 Opus. GPT Prompt Engineer also includes a Claude 3 Opus-to-Haiku conversion notebook that uses examples produced by Opus to guide Haiku generation.

Does GPT Prompt Engineer support logging integrations?

GPT Prompt Engineer supports optional Weights & Biases logging and Portkey tracing. Weights & Biases can record configurations, prompts, test cases and final ELO rankings, while Portkey logs and traces prompt chains and responses. Users enable these options through use_wandb or use_portkey.

Can GPT Prompt Engineer evaluate classification prompts?

GPT Prompt Engineer includes a classification notebook that compares generated responses with expected true or false outputs. GPT Prompt Engineer displays scores for each candidate prompt in a table, and the repository lists support for more classes as a potential contribution.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like GPT Prompt Engineer

Notebook Copilot

www.github.com

An open-source AI assistant that generates code and markdown cells in Jupyter notebooks.

Free

PromptLayer

www.promptlayer.com

Prompt management, LLM evaluations, and agent observability for engineering teams and domain experts.

Freemium

Klu.ai

www.klu.ai

Evaluate AI models on real workflows and track when new releases improve performance or cost.

Paid

Promptmetheus Forge Playground

www.promptmetheus.com

A free prompt engineering playground for composing prompts, testing outputs, and estimating inference costs.

Free