To systematically evaluate, compare, and red-team LLM applications to ensure their security and reliability.
Promptfoo provides a CLI and library for rigorously evaluating and red-teaming LLM applications. It lets developers compare different LLM models and test prompts using declarative configuration files, integrating with CI/CD pipelines. Users can quickly get started by installing it via `npm` and running `promptfoo eval` and `promptfoo view` to see results.
To systematically evaluate, compare, and red-team LLM applications to ensure their security and reliability.
Developers and MLOps engineers building LLM applications who need to rigorously test, compare, and secure their AI systems.
promptfoo is a CLI and library for evaluating and red-teaming LLM apps. Stop the trial-and-error approach - start shipping secure, reliable AI apps.
Website · Getting Started · Red Teaming · Documentation · Discord
Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the company update.
npm install -g promptfoo
promptfoo init --example getting-started