The scores get tested in public

Copying at the follower’s price: pre-registered test, published 2026-09-24, rerun 2026-09-25

A wallet’s record is scored at the prices it paid, but anyone following it buys later. This test asked whether records hold up at a later price, taken as the first trade on the same market outcome at least five minutes after the wallet’s fill, whichever side started it. Every scored fill on resolved markets was re-priced that way (512,973,581 fills in the rerun; 1.24% had no trade five or more minutes later and were left out). Returns are measured against the price curve, meaning the structural favorite-longshot premium available to anyone at that price is removed, and before fees. The thresholds and the pass rules were committed before any follower-price statistic was computed. The only follower-price results that existed then were dry runs on Sports, Esports and Tennis, and they were known when the thresholds were set.

Each wallet’s record in each category was split at the first-fill time that puts 60% of all wallet-events earlier (2026-04-23 12:36 UTC in the rerun). A wallet-category passed if its earlier record at the follower’s price sat above zero at 95% confidence over at least 20 events. The test then required, each at 95% confidence, that the passing group’s later average sit above zero and above the later average of the rest. It covered the wallet-categories with at least 20 earlier and 5 later events.

There are two runs. The first, published 2026-09-24, used data that filed most tennis and esports markets under Sports after a Polymarket tag change (see the corrections log). After the first run’s result was published, and before any rerun result existed, we recorded that a rerun with the same scripts on the corrected categories would decide the outcome and that the first run would stay published beside it (recorded 2026-09-25 03:52 UTC; the rerun started at 12:51 UTC).

In the rerun both conditions held. The passing group’s later average sat above zero, and above the rest’s. In the first run the first condition failed, because the passing group’s later average could not be told apart from zero. The rerun decides, so the evidence test passes. A second pre-registered check fell short in both runs. It put a lower bound on alerted positions under a watchlist limited to wallets whose full record holds up. In the rerun the bound came to 10.8 a week on average against the required 20, with 5 in the weakest week against the required 10. That count ranks the watchlist under scoring v1.2 (described in the validation section below and on the methodology page), because the 2026-09-25 scores had left Sports with no watchlist. We recorded that choice before the rerun started. Under the earlier scoring the rerun counted 10.0 a week with a weakest week of 5, and the first run counted 12 and 8. Under the rules committed in advance, this outcome calls for a per-wallet verdict on whether a record holds up at the follower’s price to be shown as a research label. It appears on the board, wallet pages and the wallet checker from the 2026-09-26 update, and each label shows the date of the run behind it. From 2026-09-27 to 2026-09-29 an alert from a wallet whose record did not hold up said so on its own line. That line was removed after the label test below. Which alerts go out is unchanged, and whether to limit them to wallets that hold up is left for a later review with this data.

Read plainly, records that passed came out about a tenth of a cent a share above the price curve at the follower’s price, and the rest came out most of a cent a share below it on average. Earlier records at that price sorted later results. The figures are measured against the price curve and before fees, which the venue has charged takers since February 2026, so they do not show whether buying at the follower’s price made money. The intervals treat wallet-categories as independent. Many of them trade the same events, so the true uncertainty may be wider, and the passing group’s lower bound, +0.04, is close to zero. In an exploratory cut that was not pre-registered, the rerun’s median later result was +0.12 cents a share for wallet-categories that passed, +0.07 for those whose earlier average was above zero without passing, and +0.01 for those whose earlier average was zero or below (+0.12, +0.06 and −0.07 in the first run). The two groups that did not pass averaged −0.67 and −1.01, far below their medians, which points to a tail of large later losses. The next section repeats this test at the price the next taker on the wallet’s side paid, where the passing group came out below the price curve.

Copying at a taker’s price: pre-registered test, run 2026-09-28

The test above prices a follower at the first trade on the same market outcome at least five minutes after the wallet’s fill, whichever side started it, before fees. On Polymarket a follower who buys straight away pays more than that on average. They buy from orders already on the book, at the asking price, and since February 2026 the venue has charged such takers a fee. This test prices each fill at the first trade on the same market outcome, at least five minutes after it, made by a taker on the wallet’s side. A taker here means an order that filled at once against orders already on the book (a purchase when the wallet bought, a sale when it sold). That is the price the first such taker paid. A follower’s own order could get a different one, depending on its size and timing. As in the test above, returns are measured against the price curve, so these figures do not show whether copying made money.

The rules, including how the fee would be counted, were committed on 2026-09-27 and signed at about 16:25 UTC, before the run started on 2026-09-28. When they were fixed, no figure at a taker’s price or after fees existed for any group, and the result was to be published whichever way it came out. The passing rule, the 60/40 split rule and the event minimums are the ones above, applied to the latest data, so the split falls on 2026-04-24 20:58 UTC (2026-04-23 12:36 UTC above). Passing is still judged at the price the test above uses, before fees, so the passing group is the one today’s label rule selects. The test covers wallet-categories with at least 20 earlier events and at least 5 later events that had a taker trade.

The follower’s-price rows repeat the test above on this run’s data. The rerun above found +0.12 for the passing group.

At the taker price, before fees, wallet-categories that passed on earlier events came out below the price curve on later ones, at −0.21 cents a share with an interval from −0.30 to −0.12. By the rule committed in advance, the group that today’s label rule selects did not beat the price curve at a follower’s cost on later events. The ones that did not pass came out at −1.26 (difference +1.05, 95% interval +0.95 to +1.16). Judging passing at the taker price as well selected a group that also came out below the price curve (−0.20). This is the evidence on alerts for the 2026-11-02 review of which alerts go out.

The data records a fee on most taker trades. Under the rule fixed in advance, the run could not settle how purchases are charged, so the figures above are before fees. Counted either way the rule allowed, the fee lowers the passing group’s later average to −0.42 or −0.60 cents a share.

For the passing group, the later average was +0.11 at the trade the test above uses and −0.21 at the taker trade. The first figure covers a slightly different set of wallet-categories. Across the fills that had both prices, the taker trade cost 0.33 cents a share more than the trade the test above uses. That covers the difference between the buying and selling prices on the book and any price move between the two trades.

The intervals treat wallet-categories as independent. Many of them trade the same events, so the true uncertainty may be wider. 4.9% of scored fills had no taker trade on the wallet’s side five or more minutes later and are left out. 77.2% of those won. The run did not compare that with the prices they were bought at, so it cannot say whether leaving them out raises or lowers the figures above. For 16.4% of the fills kept, the taker trade came more than an hour after the fill. Taker trades in the other outcome of the same market are not used, which lowers coverage and can push the chosen trade later. A follower of a sale is priced as the next taker seller of the same outcome. A follower with no shares to sell would buy the other outcome instead.

Across 878 graded alerted positions that had a taker purchase at least five minutes after the alert was sent (or, for a position not sent in real time, detected), the positions won 1.74 percentage points more often than that purchase’s price implied, before fees. The 95% interval runs from −0.96 to +4.44, so it includes zero. It allows for positions in the same market moving together, as on /receipts. Counting the fee either way the rule allowed, the figure is +0.95 or +1.27. 40 graded positions had no such purchase and are left out. The purchase came a median 9.8 minutes after the alert (90th percentile, 335 minutes). Figures as of 2026-09-28.

Added September 29, 2026. The same run checked a reason the methodology page gave for setting the price baseline to zero between 30¢ and 70¢, that returns there are the footprint of informed traders. The rule, fixed in advance, required scored fills whose orders took liquidity to come out more than half a cent a share above the prices paid on both sides of 50¢. They came out 2.36 cents below from 30¢ to 50¢ and 1.88 cents above from 50¢ to 70¢, so this data does not support that reading. The methodology page now says so.

Does the “didn’t hold up” label predict later results? Pre-registered test, run 2026-09-29

The research label marks a wallet-category “didn’t hold up” when its record at the 5-minute price (the follower’s price above) averaged zero or below over at least 20 events. This test asked whether records the label flags on earlier events do worse on later ones, at the taker price above, than records it does not flag. The rules were committed and signed on 2026-09-28, before the run above had produced any later-period figure. How the rules were implemented was written down after the taker-price results above were read and before this test’s. The result was read once. A first attempt stopped partway, and its partial output was deleted unread.

It covers the wallet-categories the board ranks above its baseline on their earlier events (a positive average at the prices the wallet paid), with at least 20 earlier events and at least 5 later events that had a taker trade, 1,907 flagged and 45,488 not flagged. The figures are before fees, since the run could not settle how purchases are charged. The label needed three things to count as predicting worse results. The difference, flagged minus the rest, needed a 95% interval entirely below zero. The flagged group’s average needed to be at or below zero. And a second interval, pooled over wallet-events and treating wallet-events in the same event as related, needed to sit below zero as well.

The flagged records came out at −0.78 cents a share on later events and the rest at −0.84. The difference, +0.06, has a 95% interval from −0.23 to +0.35, so the first condition failed. The other two held. The flagged average is below zero, and the pooled interval, which weights busy wallets more, put the difference at −0.26 (−0.38 to −0.13). By the rule committed in advance, the label was not shown to predict worse later results, and no change to which alerts go out is recommended on this evidence.

The rules also required six variants, reported beside the result and not part of the decision. Capping each wallet-category’s result at the 1st and 99th percentiles, the difference was +0.02 (−0.25 to +0.28). Dropping the condition that the board ranks the record above its baseline, flagged records came out 0.59 lower (−0.69 to −0.48). Measured at the 5-minute price instead of the taker price, they came out 0.36 higher (+0.09 to +0.64). Within categories, weighted by the flagged counts, the difference was +0.15. Leaving out recurring series changed nothing, because none reach this data. Records that held up (a positive earlier average at 95% confidence; 12,966 here, a different cut from the taker table’s 13,290) came out at −0.21, better than the flagged ones by 0.57 cents a share (0.27 to 0.86). Records in between (positive, short of 95%) came out at −1.09. All three groups came out below the price curve.

Validation report, published 2026-07-15

Nine quarterly cutoffs, 2024-04-01 through 2026-04-01. At each cutoff T the full scoring pipeline was refit on data resolved by T only, the board was ranked with production gates, and forward performance was measured on events resolving in the following 90 days against the as-of-T baseline. The design and the exact language rules for describing outcomes were committed before any number existed (specification 2026-07-10, reading guide 2026-07-13).

The scoring changed after this report. Since methodology v1.1 (2026-09-24), bets at 2.5 cents or less no longer count toward a score, and v1.2, applied from the 2026-09-26 update, counts each wallet’s own noise when estimating how much skill varies within a category. Every validation result on this page was computed under v1.0 and stands as published. A rerun under a later version would be published beside these results with its version stated, never in their place.

Tested strictly on data the rankings had never seen, the picture is mixed, and we publish it as it came out. Alert-eligible wallets (the wallets that met the alert watchlist’s score rules at T, before its limit of 25 per category) went on to beat the prices they paid in six of the nine windows, by about +0.7 to +1.4 cents per share with confidence intervals clear of zero (largest window n = 2,236,857 wallet-events). In the weakest window (2025-04-01) the same slice did not demonstrate forward edge beyond price at this sample size, and the top decile of the board went on to negative forward excess (-0.54 c/share, 95% CI [-0.92, -0.14], n = 73,450 wallet-events). The top decile’s interval also sits entirely below zero in the 2026-01-01 window (-0.40 c/share, 95% CI [-0.51, -0.29], n = 757,516 wallet-events; decile tables below). In that window the alert-eligible slice still went on to beat its prices (+0.82 c/share, 95% CI [+0.64, +0.98], n = 254,873 wallet-events).

Before publication, three independent layers had to agree: a timestamp-integrity audit of the resolution dating (run as a hard gate before any cutoff), a structural invariant check on the run’s output tables, and an independent recompute of both headline populations from the raw scored fills in separately written SQL. All 18 cutoff-population pairs agreed within 0.001 cents per share.

The gate slice: alert-eligible wallets

This table covers the wallets the alert rules select from. Forward price-relative excess for wallets that were alert-eligible at T, with event-clustered bootstrap intervals (events are the independence unit, and each set of mutually exclusive outcomes forms one event).

Scope, added September 27, 2026. “Alert-eligible” here means the wallets that met the watchlist’s score rules at T, which require at least 10 resolved events and a score, after small records are pulled toward their category’s event-weighted average, that is above both the as-of-T price baseline and that category average. The tested group left out wallets hidden from the public board (the list as of July 2026) and did not apply the alerts’ hand-kept exclude list. The alert watchlist takes at most 25 wallets per category, and this report did not measure that group on its own. Earlier versions of this page called the tested group the slice the paid alerts draw from. Averages are taken over wallet-events, so wallets with many events weigh more. The figures use the price each wallet paid, not the later price a follower would get (see the follower-price test above), and are before fees.

In the 2025-04-01, 2024-04-01, and 2025-01-01 windows, alert-eligible wallets did not demonstrate forward edge beyond price at this sample size; 2025-04-01 is the weakest of the three, with a negative point estimate. In the other six windows, alert-eligible wallets went on to beat the prices they paid, by about +0.7 to +1.4 cents per share, with intervals clear of zero in each.

Addendum, 2026-09-29: the watchlist’s top 25 per category

The table above covers the alert-eligible wallets, as scoped there. The alert watchlist keeps the top 25 per category by score, and the July report did not measure that group on its own. Under the commitment to publish out-of-time results either way, this re-aggregates the same run’s tables for that group, at the prices the wallets paid and before fees. Recomputed the same way, all alert-eligible wallets match the published averages and wallet-event counts at all nine cutoffs. Pooled figures weight each wallet-event equally, with a normal-approximation interval that treats positions in the same event as related (the table above used a bootstrap). Per wallet-category figures weight each wallet-category once, with an interval that treats them as independent.

Pooled, the top 25 came out above all alert-eligible wallets at eight of the nine cutoffs. The exception is 2024-04-01, the window with the fewest wallet-events. At 2024-04-01, 2024-07-01 and 2025-01-01 the pooled interval includes zero, and there the top 25 did not demonstrate forward edge beyond price at this sample size. Per wallet-category, the intervals include zero at the four earliest cutoffs, and the figures run well below the pooled ones (+4.31 against +9.62 at 2026-04-01), since the pooled figure weights busy wallets more. These are the prices the wallets paid. No test above isolates the top 25 at a follower’s price.

Decile tables

Forward excess by as-of-T rank decile. Under the pre-registered reading guide, the only licensed pairwise comparison is decile 1 against decile 10, and only where their intervals do not overlap; where the top decile’s interval includes zero, the mandatory sentence appears verbatim. Patterns the guide has no language for stay in the tables.

Rank correlation

Spearman between as-of-T rank and mean forward excess, wallets with at least 10 forward events. Point estimates only; no interval is computed for rho, so none is implied. At every cutoff the direction was the same: higher-ranked wallets went on to higher forward excess.

Persistence

Share of top-300 wallets at T still top-300 at the next cutoff, against the base rate of being ranked top-300 at all. Descriptive only; no interval accompanies this table, so no comparison adjectives are used. The first transition is degenerate: the early board ranked fewer wallets than the top-300 floor.

Riders

Estimation-population rider: dropped, pre-registered. Recorded 2026-07-11, before any cutoff ran: excluded market-maker and bot wallets shift no price-baseline bucket by more than 0.01 cents and no category prior moment by more than 0.03 cents, so the with-and-without grid was dropped in advance, and that decision was published on this page before the run.

Resolution-timestamp robustness rider: kept. Timestamp error matters to this design only where it could move a fill across a window edge. The rider recomputes both headline populations with every fill resolving within 24 hours of either window edge removed. Across all 18 pairs it moves the mean by at most 0.33 cents per share (the grid’s smallest-n negative window, where the rider moves it toward zero), and by at most 0.06 in every window with more than 100,000 wallet-events. No sign that survives an interval in the main grid changes under the rider.

Limitations

• Resolution dating. The windows slice on each market’s recorded close moment. An audit of 760,617 resolved markets with trades found 1.67% dated more than 10 minutes after the best available truth proxy and 0.21% more than 24 hours after it (median lateness 0.59 hours, p90 30.4 hours, dominated by sports). What matters to the design is boundary crossings: 271 events (0.036%) span a window edge closely enough for this error to move them between windows; the rider above bounds the effect. A further 2,107 resolved markets carry a scheduled-close fallback date in the future; they are excluded from every window by construction, which loses sample but cannot leak the future into a ranking.
• Survivorship. Forward windows only contain wallets that kept trading. A wallet that stopped after T contributes nothing forward, and stopping is not random.
• Category mix shift. The venue’s category composition moved across the nine windows; a rank built on one mix is tested against another. The two windows overlapping major sports cycles are the grid’s largest by an order of magnitude.
• Fees. Returns are price-relative and fee-exclusive throughout.
• Retagging churn. The venue re-tags markets; category-level slices inherit that churn.
• High-frequency series families are excluded from scoring and from this validation, per the standing methodology.

Already public

• The methodology: price-relative scoring, empirical-Bayes shrinkage, independence rules, and a limitations section that states what the scores cannot yet claim.
• Calibration diagrams for each category’s #1, open on its wallet page in the category it leads and computed from the same scored trades the rankings use.
• The receipts ledger: every alerted position, recorded at detection and graded on resolution.
• The statistics page: aggregate calibration, alert grading, and trader-skill numbers from the same scored data, refreshed with the board.