Prompt testing is the practice of systematically evaluating how different large language models (LLMs) respond to the same or similar inputs. Just as software engineers write unit tests for code, AI engineers write and refine prompts to ensure models behave reliably across a range of inputs.

LLM Test Bench lets you configure multiple AI providers, run prompts against them, compare outputs side-by-side (Duel mode), and build a searchable history of every experiment — so your learning compounds over time.

The same question asked to different models — or even to the same model with a slightly different system prompt — can produce dramatically different results. Prompt testing helps you:

• Find the best model for your task — A coding assistant, a creative writer, and a factual Q&A bot all have different ideal models.
• Reduce costs — Smaller, cheaper models often perform just as well for simple tasks. Testing reveals where you can save.
• Improve reliability — A well-crafted system prompt that's been tested reduces hallucinations and keeps the model on-task.
• Document your findings — The History tab turns ephemeral experiments into a searchable knowledge base.
• Avoid vendor lock-in — By testing multiple providers, you understand trade-offs and can switch if pricing or availability changes.

Step 1 — Add a Provider

Go to Configurations and click + Add Provider. Type a provider name — suggestions from a built-in catalog will appear automatically. Selecting a known name (e.g. "Anthropic") auto-fills the API URL.

Step 2 — Add Models

In the Models field, type or pick from the suggestions and click Add. Star (☆) a model to mark it as the default — it will be pre-selected whenever you open the Test tab.

Step 3 — Run a Test

Go to Test, choose your provider and model, enter a System Prompt and User Message, then click ▶ Run Test. The output and token counts appear below and are saved automatically to History.

Popular Provider URLs

The three parameters in the Test tab control how the model generates text. They are collapsed by default — click ⚙ Parameters to expand them.

Temperature — Controls randomness

Max Tokens — Maximum output length

Top P (Nucleus Sampling) — Vocabulary diversity

Below are ready-to-use system prompts for common tasks. Copy them into the System Prompt field to get started quickly.

General Assistant

Code Review

Creative Writing Coach

Data Analyst

Sample Duel prompts — try these in the ⚔️ Duel tab to compare models:

• Consistency test: "Explain quantum computing in simple terms." — reveals explanation style differences.
• Creativity test: "Write a haiku about the feeling of debugging code at 2am."
• Reasoning test: "A bat and ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost?"
• Code test: "Write a Python function to find all prime numbers up to n using the Sieve of Eratosthenes."

Recommended models by task