Emergence, and the Trouble with Thresholds

1The claim

The observation that started this was straightforward. Plot a model's performance on certain tasks against its scale, and for some tasks the curve is flat at chance across several orders of magnitude and then rises sharply. The ability appears absent, absent, absent, and then present. Multi-step arithmetic, certain kinds of instruction following, and some symbolic manipulation tasks all showed this shape.

The word attached to it was emergence, borrowed from complex systems, where it names properties of a whole that are not properties of its parts. That borrowing carried freight. In physics a phase transition is a specific mathematical object with an order parameter and a critical point; in casual usage emergence means something more like 'surprising', and the two senses got mixed almost immediately.

The stakes were never purely academic. If capabilities genuinely appear discontinuously at unpredictable scales, then evaluation of a smaller system tells you little about a larger one, and safety arguments that rely on testing before scaling are undermined. That is why this particular methodological dispute ended up in policy documents.

2The deflationary reply

The strongest criticism is elegant and largely correct. Many of the tasks showing sudden onset were scored with discontinuous metrics: exact string match, or all-or-nothing multi-step accuracy. Under such a metric a model that is steadily improving registers nothing until it crosses the threshold where the whole output is right.

Consider five-digit addition scored by exact match. A model whose per-digit accuracy climbs smoothly from 0.7 to 0.95 goes from getting roughly seventeen per cent of whole answers right to roughly seventy-seven per cent. The underlying improvement is smooth. The reported curve is a cliff, and the cliff is the metric's, not the model's. Rescore the same outputs with a continuous measure — per-token likelihood, edit distance, partial credit — and in many cases the sharpness dissolves into an ordinary smooth curve.

There is a second artefact worth naming: sparse sampling. Model sizes are not sampled evenly; they cluster at a few scales because those are the ones anyone trained. With four points on a log axis, a smooth sigmoid and a step function are not distinguishable, and the eye supplies the step.

3Where the deflation stops

Two things survive the critique and are usually skipped over by people who enjoyed it too much.

First, discontinuous metrics are sometimes the right metrics. If a system must produce a correct SQL query, a compiling program, or a valid legal citation, partial credit is a researcher's convenience and not a user's experience. A query that is nearly right returns wrong data with full confidence. The smooth underlying curve is real and it is also, for the purpose of deciding whether to deploy, not the relevant curve. Both statements hold at once.

Second, the reply explains the shape of the curve without explaining its location. Even granting that improvement is smooth, the scale at which a task becomes usably reliable is not currently predictable in advance. We can extrapolate loss with reasonable confidence — scaling laws are among the better-behaved empirical regularities in the field — and we remain poor at predicting which downstream competences a given loss will buy. That prediction gap is the practically important claim, and it is untouched by the metric argument.

4What would settle it

The cleanest test is prospective and it is expensive, which is why it is rare. Before training a larger model, publish predictions: at this scale, this task will reach this accuracy, under a continuous metric fixed in advance. Then train and check. A few groups have done versions of this and the results are mixed in an informative way — aggregate loss predictions hold up well, specific downstream task predictions much less so.

The obstacle is not conceptual. It is that scaling runs are enormously expensive, that commercial labs have no incentive to publish predictions they might miss, and that the sample size available to anyone is small. This is a field where the crucial experiment costs more than most departments' annual budget, and that shapes what gets known.

Absent that, we are largely arguing about how to draw curves through four points, and both sides are doing it with more confidence than four points support.

5Why the word matters

I would retire the term, and not for pedantic reasons. Emergence imports a set of associations from statistical physics — genuine criticality, universality, order parameters — that have not been demonstrated here and probably do not apply. Using the word smuggles in the claim.

It also does rhetorical work in two opposite directions, which is a reliable sign that a term has stopped carrying information. For those forecasting rapid capability gain, emergence implies unpredictable jumps and therefore urgency. For sceptics, demonstrating that a particular curve was a scoring artefact is taken to deflate the entire enterprise. Neither inference follows, and both are made routinely.

The precise claims are available and they are less quotable. Some task curves are sharp under discontinuous scoring and smooth under continuous scoring. Aggregate loss is predictable from scale within useful bounds. The mapping from loss to specific downstream competence is not currently predictable. Say those three things and there is very little left to argue about, which may be why nobody says them.

6What I take from it

This dispute is, to me, the best available case study in how the field handles evidence. A striking empirical claim was made on reasonable grounds; a sharp methodological critique showed much of it to be an artefact; the critique was then overextended into a general dismissal; and the residual, genuinely unresolved question — can we predict downstream capability from scale? — received considerably less attention than either of the louder positions.

That pattern is not unique to emergence and I expect to write about it again. The useful discipline is to keep asking what precisely is claimed, what measurement would distinguish it from its negation, and whether that measurement has been made. In this case the answer to the third question is: partly, at small sample size, by people with an interest in the result.

Which is not a scandal. It is what an immature empirical science looks like from inside, and the appropriate response is neither excitement nor contempt but patience — a quality in conspicuously short supply on all sides of this.