Skip to main content

Automation in Promptscape

Promptscape provides tools to accelerate and automate your prompt engineering workflows in VSCode.

Scenarios​

Scenarios are combinations of prompts & past messages. They are useful when you need to see how your virtual agent behaves when a conversation is continued, rather than started. Scenarios are useful for verifying things like:

  • Proper summary of RAG knowledge, in the context of a broader chat history
  • Function calls / tools calls where the data is provided in multiple message turns
  • Any chat testing scenario that requires both a system prompt and user/assistant messages to be included in the test

Runbooks​

When you want to test a prompt by sending multiple messages in sequence, Promptscape Runbooks are the solution. With runbooks, you can:

  • Run a runbook completely, or step through message-by-message
  • Express testing conversations as JSON, and commit to scm for sharing & versioning
  • Save development time by easily executing the same inputs acorss models, inference parameter combinations, and prompt versions