Create and Manage Prompts Easily — Scorecard AI

When it comes to large language models (LLMs), a prompt refers to the specific instructions a user provides to produce an expected output.

The Challenge of Many Prompts

It’s crucial to recognize that some prompts will be more effective than others. In fact, it may take experimenting with dozens or even hundreds of prompts before finding the optimal prompt that generates satisfying results and scores well based on your defined metrics. As you can imagine, managing a large number of prompts, or numerous variations of the same prompt, can become quite complex.

Easy Prompt Management With Scorecard

Scorecard makes the process easy to create and manage multiple prompts and versions of the same prompt without getting lost. Check out the features that Scorecard offers in terms of Prompt Management!

Get an All-in-One Overview of Your Prompts in the Prompts View

The Prompts view is where you can find all your generated prompts in one place. Use the search bar to find prompts, or sort prompts by date or name. For each individual prompt, you have the option to get more details of important prompts or archive the ones you do not need anymore.

Create Prompts Easily

To create a prompt from the Prompts view, simply click on the New Prompt + button. The Scorecard UI assists you in the process of prompt creation and asks you for the necessary information, such as prompt metadata, model with model parameters, and the prompt messages.

You are not sure which model to choose from? Scorecard is there to help and provides background information about each model in interactive info boxes.

Interactive Model Background Information in Prompts Creation

Easily break up the different prompt messages by specifying the type of prompt message (you can choose between System, User, and Assistant).

Split up Prompt Messages in Prompts Creation

Prompt Creation and Selection in the Playground

In case you are in the Playground, you can either:

Directly create a new prompt by filling out the user fields

Scorecard Playground: Creating a New Prompt

Select a saved prompt with the interactive drop-down

Scorecard Playground: Searching a Saved Prompt

Uniquely Identify Your Prompts To Quickly Find Them

Each prompt in Scorecard receives a unique Prompt ID. This unique identification might be useful in the following cases:

Identify top-performing prompts by analyzing the results of runs using the same Prompt ID
Quickly find prompts with the Prompt ID in the Prompts view or the Playground

Version Your Prompts and Follow Changes You Have Made to the Same Prompt

As mentioned, each prompt receives a unique Prompt ID. However, you might have the situation where you iteratively optimized the same prompt by making changes in each round. Scorecard makes it easy to organize these differentiating versions into the same prompt. With Scorecard’s Prompt Versioning, you can see the different versions of the same prompt and easily identify which version was published to the Production system.