Leaderboard

The primary metric is VUS-PR (Paparrizos et al., VLDB 2022). PA-F1 is not shown by default (see ch07 for why). The config_hash of every row can be resolved back to its full configuration in benchmarks/results/*.json.

• Current numbers: lite leaderboard table (auto-generated from the lite profile)
• The charts below are auto-generated from the results parquet by tsad-forge viz.

Performance by generation — does newer mean better?

Leaderboard table (metric selector)

Model × dataset heatmap (VUS-PR)

Critical difference diagram

Performance vs cost

Metric divergence — evidence of PA inflation

Case viewer

Dataset statistics