Labeling Volume

50 Rows at 96%, Ship It? Size Labeling by the CI Lower Bound, Not the Pretty Number

"We labeled 50, 96% correct — ship it?" No — the statistical lower bound is only 86%. When I designed the labeling gate for a customer-service agent, the first rule I pinned was: read the confidence-interval lower bound, not the point estimate. In 5 minutes you'll see through the small-sample 96% mirage; in 10 you'll have a "true accuracy → rows to label" table that explodes near the threshold; in 20, a CI stopping rule — label more where it's needed, don't waste it where it's done, and call a stop when the answer is fix-the-agent.