How Much to Label: Not a Percentage of Traffic, but "Label Until You Can Conclude"
"We labeled 50, 96% correct — ship it?" No — the statistical lower bound is only 86%. In 5 minutes you'll see through the small-sample 96% mirage; in 10, why labeling volume tracks "intents × channels," not traffic share; in 20, you'll have a table mapping true accuracy to rows-to-label, plus the cheapest rule there is: labeling volume grows with the number of intents × channels, not with traffic.
Jul 3, 2026·12 min read