Skip to main content

One post tagged with "threshold"

View All Tags

Autonomous trading: sample size sets the bar, not the number

· 20 min read
Vadim Nicolai
Senior Software Engineer

The largest spread-to-standard-error ratio on the board is 4.2996, and it is not significant. It has to clear 4.302653 — the two-sided 5% critical value on 2 degrees of freedom. That is not the 1.959964 the rest of the one-day table is measured against. 4.2996 falls 0.003 short. Nothing about the number is wrong; it was judged against a bar that belonged to a different sample size — the error Bailey and López de Prado (2014) built the deflated Sharpe ratio to catch, where the length of the track record behind a statistic is part of the threshold and not a footnote to it.

That gap is worth a long article not because 0.003 is large, but because the board prints no column that says so, and because the system that produced the number declined to promote, demote or score anything on the strength of it. The failure mode has a name. Bailey and López de Prado (2014) describe it as an undeflated ratio: a performance statistic reported without controlling for the number of trials behind it, the length of the track record, and the non-normality of the sample. The correction applied to this cell is the crudest possible version of their adjustment — the one that comes free with a t-table.