Jul 4, 2026Your Labels Are Your Ceiling — One "Swap Half a Size Up" Gets Three Answers From Two AgentsA customer's one line — "I want to swap half a size up" — hides four calls: exchange or return, intercept the shipment or not, refund the price difference or not, and against which price. Two skilled agents label the same 50 rows back-to-back and agree on only 35 — your 96% accuracy was measured with a ruler that's only 70% self-consistent. In 20 minutes you'll have a flow for measuring agreement first, then writing rules like "which price the difference is refunded against" into a rubric.After Launch · 13 min read · EN · 中
Jul 3, 202650 Rows at 96%, Ship It? Size Labeling by the CI Lower Bound, Not the Pretty Number"We labeled 50, 96% correct — ship it?" No — the statistical lower bound is only 86%. When I designed the labeling gate for a customer-service agent, the first rule I pinned was: read the confidence-interval lower bound, not the point estimate. In 5 minutes you'll see through the small-sample 96% mirage; in 10 you'll have a "true accuracy → rows to label" table that explodes near the threshold; in 20, a CI stopping rule — label more where it's needed, don't waste it where it's done, and call a stop when the answer is fix-the-agent.After Launch · 12 min read · EN · 中
Jul 2, 2026The 96% Turned Red the Moment We Split It by Channel — Two Fatal Failures of Eval SamplingThe vendor drew 200 rows at random and reported "96% accuracy." I split those 200 by channel — in the small public-domain channel, the price-difference cases were nearly all wrong, and the high-traffic private channel had diluted it into a pretty number. In 5 minutes you'll see through both failures of random sampling (missing the tail + a big channel drowning a small one); in 20 you'll have a stratified frame with channel drill-down and risk-layer over-sampling.After Launch · 13 min read · EN · 中