Walk-Forward / Out-of-Sample Validation

Does the gate hold out of sample? Train on earlier weeks, freeze the rules, test on later weeks — does each rule keep its sign? · Built 2026-06-09 · Machine summary: vwap_pull_in_walkforward_oos_summary.json
Split: plain vwap-pull-in n=132 · TRAIN 2026-04-10→05-13 (n=77) · TEST 2026-05-14→06-04 (n=55) · split date 2026-05-14 · no DB query.
MIXED — do not promote on this OOS alone; forward-test required

The post-entry fade-exit and the gap-RED / avoid-bad-type rules keep their sign out of sample. But "megacap-tech = profitable" and the entry-gate-alone FLIP to mildly negative in the test half — they're relative (least-bad) filters, not absolute edges. The test window was a much worse regime (baseline +$91 → −$389), cells are tiny (full-clean test n=7), and one IBIT trade (−$216) is >half the test loss. Verdict: durable core confirmed; promote nothing yet — run a live forward-test.

⚠ Two things dominate the test half — read the signs, not the levels

(1) Regime shift: plain Pull-In went from +$91.08 / 41.6% (train) to −$389.21 / 27.3% (test). The later weeks were simply a bad stretch for this setup; no static gate fixes that. (2) One outlier: the IBIT 6/3 trade (−$216.15, in the test half) is >55% of the entire test-half loss — ex-outlier the test baseline is −$173.06. So judge rules by whether their sign/separation persists, not by absolute P&L.

Rule-by-rule: does the sign hold out of sample?

RuleTrain avg (n)Test avg (n)Holds OOS?
Cut the post-entry fade (fade @5min)−$2.33 (28)−$19.54 (23)holds — strongest fade loses both halves; 8.7% win in test
Keep not-fades+$3.19 (49)+$0.38 (31)holds positive/flat both
Pass \|gap\|>1.5% (RED)−$7.80 (31)−$4.35 (23)holds by win rate 25.8%→21.7%; PnL outlier-contaminated*
Avoid non-megacap types−$2.56 (47)−$9.70 (34)holds negative both
"Megacap-tech = profitable"+$7.05 (30)−$2.83 (21)FLIPPED relative-best only, not absolute
Entry gate alone (megacap & \|gap\|≤1.5)+$12.23 (21)−$2.95 (16)FLIPPED degrades in bad regime
Full stack (gate & cut fade)+$13.57 (15)+$6.15 (7)holds test n=7 fade-exit rescues the gate
Early expansion @5min+$10.99 (5)+$0.81 (5)holds tiny n

*In the test half, the low-gap bucket (−$9.03 avg) looks worse than gap>1.5 (−$4.35) only because the IBIT −$216 outlier sits in the low-gap bucket. By win rate, gap>1.5 is still clearly worse (21.7% vs 31.2%).

The key insight: the fade-exit rescues the gate OOS

The entry gate alone (megacap + controlled gap) flipped negative in test (−$2.95). Adding the post-entry rule "cut the fade" turned that same population into +$6.15 in test (full stack). In a bad regime, the entry filter isn't enough — the post-entry fade-exit is what preserves the edge. This is the most important OOS finding: the durable value is in management, not just selection. (Test n=7 — encouraging, not proven.)

What held vs what didn't

Held out of sample

Did NOT hold out of sample

Recommendation

HOLD promotion. The walk-forward did its job: it confirmed the durable core is the post-entry fade-exit + gap-RED + avoid-bad-type, and exposed "megacap = money" as partly favorable-regime luck. The test half is too regime- and outlier-dominated to settle promotion on history. Next step is a LIVE forward-test (the forward_test_live_log already exists) — not another historical slice. Promote nothing to production yet.

Validation status

Sample-size & method caveats

Source & related

Read-only out-of-sample validation · computed from on-disk Liberator artifacts, no DB · no edits to /strategies/, /scorecards/, gen_scorecard.py, _scientific_overlay.js, taxonomy, or DB · no Stage B reclassification · nothing promoted to production.