PromptWizard
StaleDescription
A task-aware agent-driven prompt optimization framework from Microsoft Research that iteratively refines prompts for better LLM performance.
Key Features
- Self-evolving prompt optimization where the LLM generates, critiques, and refines its own prompts iteratively
- Feedback-driven refinement combining positive, negative, and synthetic examples with Chain of Thought steps
- Supports three scenarios: no examples, synthetic examples, and training data-driven optimization
- Custom dataset support via JSONL format with configurable hyperparameters for mutations and refinement steps
- Pre-built demos for GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) benchmark datasets
Use Cases
💡 Automatically improving LLM task performance by optimizing prompts on custom datasets
💡 Reducing manual prompt engineering effort through iterative self-critique and refinement
💡 Research teams benchmarking prompt optimization techniques across multiple datasets
💡 Production teams fine-tuning prompt quality for domain-specific classification or reasoning tasks
Categories
Quick Start
git clone https://github.com/microsoft/PromptWizard && cd PromptWizard && pip install -e . Configure .env with OPENAI_API_KEY or AZURE_OPENAI_ENDPOINT, set promptopt_config.yaml, then run the demo notebooks in demos/ folder. Supports GSM8k, SVAMP, AQUARAT, and custom JSONL datasets.