Analysis / Guide

A short winning run cannot validate every decision behind it

Outcomes are noisy, and a useful process can still have disappointing results. Equally, poor reasoning can produce winners. Review the original information, decision rule and sample before changing a method. Keep outcome measures separate from the assessment of whether the planned process was followed.

Three checks worth keeping

  • Retain the original decision rule through the review window.
  • Report the full run of outcomes rather than highlights.
  • Compare process errors with result errors separately.

Put the question to work

Stability of the evaluation rule helps learning; it does not justify persisting with a demonstrably broken method.

Use the strike rate & uncertainty to organise the next step, then explore the worked example.

Using this guide

The name is not a promise of winning selections or financial returns.

Continue your racing research

For a related racing resource, explore PTPTips horse racing ratings and form. Check the source for changing race information.

Terms used in this guide

Denominator
The count of opportunities used to form a rate.
Coverage
The population and events for which a method actually supplies an output.
Selection bias
Distortion associated with how observations enter the analysed sample.
Prospective record
A record made before the outcome it will later be evaluated against.
Counterargument
A plausible alternative explanation that can be compared with the preferred one.
Reproducibility
Enough documented inputs and rules for another person to reconstruct the analysis.