The Best Rule in Machine Learning Was Measured with a Crooked Ruler

`★ Insight ─────────────────────────────────────` This is a prose line-edit, not a code task — so I'll work directly rather than dispatch agents. The flagged tells here cluster in three spots: one "quietly" (para 3), three honesty-family words (paras 4 and 6), and one negative-parallelism construction (para 4). `─────────────────────────────────────────────────`

There is a piece of folk wisdom that nearly everyone who trains artificial intelligence has learned to trust. Feed the system its training data in small bites — a few dozen examples at a time, rather than enormous gulps — and the finished system tends to do better out in the world, on examples it has never seen. It works so reliably that it became a habit before anyone could fully say why.

The favorite explanation borrowed a picture from physics. Training a network means hunting for the lowest point in a vast invisible landscape, where low ground means few mistakes. Some low points sit at the bottom of wide, gentle valleys; others sit in narrow spikes. The wisdom went: small bites settle you into the wide valleys, and a valley is forgiving. The real world will always be a little different from your training data, nudging you off the exact bottom — and in a broad valley a nudge barely matters, while on a spike it sends your answers off a cliff. Small bites, flat valleys, sturdy intelligence. The story held up a decade of theory.

Then, in 2017, a group of researchers pointed out something devastating. You could take any one of these comfortable valleys and make it look like a vicious spike — without changing a single prediction the network made. The trick was just relabeling the math, rescaling the numbers the network is described with. Same machine, same answers, different-looking landscape. It was like measuring a room in feet, then measuring again in inches and announcing the room had grown. The number changed; the room did not. The standard yardstick for flatness, it turned out, was measuring how the network happened to be written down, not anything true about it. The beloved intuition was resting on a ruler that bent whenever you wanted it to.

The new work fixes the ruler. Instead of measuring the steepness of the valley in raw, arbitrary units, it weights every measurement by how sensitive the network's actual behavior is in each direction — a built-in conversion table that keeps the reading true no matter what units you started in. Measured this way, flatness stops being a mirage. And the payoff is larger than the original claim: the authors prove that small-bite training is structurally pulled toward flat valleys, not stumbling into them by luck, and now the pull is stated in terms nobody can fake by relabeling.

The numbers are stark. On a standard handwriting task, a modest network trained in bites of 32 lands in a valley scoring about 388 on the corrected ruler. Widen the bites to 512 and the score explodes to roughly 71,600 — a far sharper, more fragile perch — while accuracy slips. The culprit is counterintuitive: big gulps give the training a cleaner, steadier sense of direction, which sounds like help. But the jitter of small bites was the very thing jostling it away from the spikes. The noise was doing the work.

The fix comes with a caveat. The clean math needs conditions real training only approximates, and the tested networks are tiny beside today's giants. The ruler is fixed. The largest rooms remain unmapped.