Probability of backtest overfitting (PBO)

An estimate of how likely an in-sample strategy selection is to underperform out of sample.

The probability of backtest overfitting estimates how often the strategy selected as best in one part of a historical sample would perform relatively poorly in the complementary, unseen part. The proposed method uses combinatorially symmetric cross-validation (CSCV) across a set of candidate strategies or parameter configurations.

A high PBO warns that selection may be exploiting noise rather than finding a repeatable edge. The estimate applies to the full selection exercise, so omitting discarded configurations or repeatedly reusing the same data can make the reported result misleading.