Skip to content
Documentation
Cookbook
Quality & Evals: Testing Data and Tool Evaluation

Quality & Evals: Testing Data and Tool Evaluation

Cookbooks is valuable not just for runnable examples, but also for “how to evaluate and align quality” with concrete, executable recipes.

Recommended notebooks

1) Building evals

Building evals
Core eval components (code/human/model)

2) Synthetic test cases

Generate synthetic test cases
Generate test data for prompt templates

3) Tool evaluation

Tool evaluation
Evaluate tool calling behavior and loops

4) Quality loops inside agents (optional)

Evaluator-Optimizer lives under Agent Patterns:

  • ../agent-patterns/evaluator-optimizer

How this fits repo guardrails

  • Structure + reproducibility: make test-notebooks (fast, CI-friendly)
  • Execution + regression: make test-notebooks-exec (slow, periodic)
Start with a minimal eval, then iterate on metrics, coverage, and failure diagnosis.