Back to Details

dspy-miprov2-optimizer

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

DSPy MIPROv2 Optimizer

DSPy MIPROv2 Optimizer

Goal

目标

Jointly optimize instructions and few-shot demonstrations using Bayesian Optimization for maximum performance.
通过贝叶斯优化（Bayesian Optimization）联合优化指令和少样本演示，以实现最高性能。

When to Use

适用场景

  • You have 200+ training examples
  • You can afford longer optimization runs (40+ trials)
  • You need state-of-the-art performance
  • Both instructions and demos need tuning
  • 您拥有200+训练示例
  • 您可以接受较长时间的优化运行（40+次试验）
  • 您需要达到最先进的性能水平
  • 指令和演示都需要调优

Related Skills

相关Skill

  • For limited data (10-50 examples): dspy-bootstrap-fewshot
  • For agentic systems: dspy-gepa-reflective
  • Measure improvements: dspy-evaluation-suite
  • 数据有限时（10-50个示例）：dspy-bootstrap-fewshot
  • 智能体系统场景：dspy-gepa-reflective
  • 性能评估：dspy-evaluation-suite

Inputs

输入参数

InputTypeDescription
program
dspy.Module
Program to optimize
trainset
list[dspy.Example]
200+ training examples
metric
callable
Evaluation function
auto
str
"light", "medium", or "heavy"
num_trials
int
Optimization trials (40+)
输入类型描述
program
dspy.Module
待优化的程序
trainset
list[dspy.Example]
200+训练示例
metric
callable
评估函数
auto
str
可选值："light"、"medium"或"heavy"
num_trials
int
优化试验次数（40+）

Outputs

输出结果

OutputTypeDescription
compiled_program
dspy.Module
Fully optimized program
输出类型描述
compiled_program
dspy.Module
完全优化后的程序

Workflow

工作流程

Three-Stage Process

三阶段流程

  1. Bootstrap - Generate candidate demonstrations
  2. Propose - Create grounded instruction candidates
  3. Search - Bayesian optimization over combinations
  1. 引导生成（Bootstrap） - 生成候选演示样本
  2. 候选提出（Propose） - 创建基于实际场景的指令候选
  3. 搜索优化（Search） - 对组合方案进行贝叶斯优化

Phase 1: Setup

阶段1：环境设置

python
import dspy
from dspy.teleprompt import MIPROv2

lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)
python
import dspy
from dspy.teleprompt import MIPROv2

lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

Phase 2: Define Program

阶段2：定义程序

python
class RAGAgent(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought("context, question -> answer")
    
    def forward(self, question):
        context = self.retrieve(question).passages
        return self.generate(context=context, question=question)
python
class RAGAgent(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought("context, question -> answer")
    
    def forward(self, question):
        context = self.retrieve(question).passages
        return self.generate(context=context, question=question)

Phase 3: Optimize

阶段3：执行优化

python
from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(
    metric=dspy.evaluate.answer_exact_match,
    auto="medium",  # Balanced optimization
    num_threads=24
)

compiled = optimizer.compile(RAGAgent(), trainset=trainset)
python
from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(
    metric=dspy.evaluate.answer_exact_match,
    auto="medium",  # 平衡型优化配置
    num_threads=24
)

compiled = optimizer.compile(RAGAgent(), trainset=trainset)

Auto Presets

Auto预设配置

PresetTrialsUse Case
"light"
~10Quick iteration
"medium"
~40Production optimization
"heavy"
~100+Maximum performance
预设值试验次数适用场景
"light"
~10次快速迭代验证
"medium"
~40次生产环境优化
"heavy"
~100+次追求极致性能

Production Example

生产环境示例

python
import dspy
from dspy.teleprompt import MIPROv2
from dspy.evaluate import Evaluate
import json
import logging

logger = logging.getLogger(__name__)

class ReActAgent(dspy.Module):
    def __init__(self, tools):
        self.react = dspy.ReAct("question -> answer", tools=tools)
    
    def forward(self, question):
        return self.react(question=question)

def search_tool(query: str) -> list[str]:
    """Search knowledge base."""
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [r['long_text'] for r in results]

def optimize_agent(trainset, devset):
    """Full MIPROv2 optimization pipeline."""
    
    agent = ReActAgent(tools=[search_tool])
    
    # Baseline evaluation
    evaluator = Evaluate(
        devset=devset,
        metric=dspy.evaluate.answer_exact_match,
        num_threads=8
    )
    baseline = evaluator(agent)
    logger.info(f"Baseline: {baseline:.2%}")
    
    # MIPROv2 optimization
    optimizer = MIPROv2(
        metric=dspy.evaluate.answer_exact_match,
        auto="medium",
        num_threads=24,
        # Custom settings
        num_candidates=15,
        max_bootstrapped_demos=4,
        max_labeled_demos=8
    )
    
    compiled = optimizer.compile(agent, trainset=trainset)
    optimized = evaluator(compiled)
    logger.info(f"Optimized: {optimized:.2%}")
    
    # Save with metadata
    compiled.save("agent_mipro.json")
    
    metadata = {
        "baseline_score": baseline,
        "optimized_score": optimized,
        "improvement": optimized - baseline,
        "num_train": len(trainset),
        "num_dev": len(devset)
    }
    
    with open("optimization_metadata.json", "w") as f:
        json.dump(metadata, f, indent=2)
    
    return compiled, metadata
python
import dspy
from dspy.teleprompt import MIPROv2
from dspy.evaluate import Evaluate
import json
import logging

logger = logging.getLogger(__name__)

class ReActAgent(dspy.Module):
    def __init__(self, tools):
        self.react = dspy.ReAct("question -> answer", tools=tools)
    
    def forward(self, question):
        return self.react(question=question)

def search_tool(query: str) -> list[str]:
    """搜索知识库。"""
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [r['long_text'] for r in results]

def optimize_agent(trainset, devset):
    """完整的MIPROv2优化流程。"""
    
    agent = ReActAgent(tools=[search_tool])
    
    # 基线性能评估
    evaluator = Evaluate(
        devset=devset,
        metric=dspy.evaluate.answer_exact_match,
        num_threads=8
    )
    baseline = evaluator(agent)
    logger.info(f"基线性能: {baseline:.2%}")
    
    # MIPROv2优化
    optimizer = MIPROv2(
        metric=dspy.evaluate.answer_exact_match,
        auto="medium",
        num_threads=24,
        # 自定义配置
        num_candidates=15,
        max_bootstrapped_demos=4,
        max_labeled_demos=8
    )
    
    compiled = optimizer.compile(agent, trainset=trainset)
    optimized = evaluator(compiled)
    logger.info(f"优化后性能: {optimized:.2%}")
    
    # 保存优化结果及元数据
    compiled.save("agent_mipro.json")
    
    metadata = {
        "baseline_score": baseline,
        "optimized_score": optimized,
        "improvement": optimized - baseline,
        "num_train": len(trainset),
        "num_dev": len(devset)
    }
    
    with open("optimization_metadata.json", "w") as f:
        json.dump(metadata, f, indent=2)
    
    return compiled, metadata

Instruction-Only Mode

仅指令优化模式

python
from dspy.teleprompt import MIPROv2
python
from dspy.teleprompt import MIPROv2

Disable demos for pure instruction optimization

禁用演示样本优化，仅优化指令

optimizer = MIPROv2( metric=metric, auto="medium", max_bootstrapped_demos=0, max_labeled_demos=0 )
undefined
optimizer = MIPROv2( metric=metric, auto="medium", max_bootstrapped_demos=0, max_labeled_demos=0 )
undefined

Best Practices

最佳实践

  1. Data quantity matters - 200+ examples for best results
  2. Use auto presets - Start with "medium", adjust based on results
  3. Parallel threads - Use 
    num_threads=24
    or higher if available
  4. Monitor costs - Track API usage during optimization
  5. Save intermediate - Bayesian search saves progress
  1. 数据量至关重要 - 200+示例才能获得最佳效果
  2. 使用Auto预设 - 从"medium"开始，根据结果调整
  3. 多线程并行 - 若资源允许，使用
    num_threads=24
    或更高配置
  4. 监控成本 - 优化过程中跟踪API调用量
  5. 保存中间进度 - 贝叶斯搜索会自动保存优化进度

Limitations

局限性

  • High computational cost (many LLM calls)
  • Requires substantial training data
  • Optimization time: hours for "heavy" preset
  • Memory intensive for large candidate sets
  • 计算成本高（需要大量LLM调用）
  • 需要充足的训练数据
  • 优化时间："heavy"预设需耗时数小时
  • 处理大规模候选集时内存占用高

Official Documentation

官方文档