The Benchmark Number Is a Burst — I Ran a Fanless M1 Flat Out for 30 Minutes

Every local-inference benchmark on this site so far, including my own, has measured the same thing: a handful of generations with idle gaps between them, on a machine that started cold. That is a burst. It is a fine way to compare two models, and it is a poor description of what a local assistant actually does to a laptop, which is generate more or less continuously for as long as you are working.

The gap matters more on this machine than most, because the MacBook Air M1 has no fan. There is nothing to spin up when the chassis warms — the SoC either sustains its clocks passively or it doesn't. The folklore says it doesn't, and that a fanless Mac falls off a cliff under sustained load. I had never measured it, and neither had the two posts I've already published off this laptop.

So I ran it flat out for thirty minutes.

How the run was built

One model, llama3.2:1b-instruct-q4_K_M, generating back to back with no idle gaps for 30 minutes of wall clock — 424 completed generations. Every one of them is the same unit of work: the same prompt, seed 42, temperature 0, and num_predict: 200, so each generation produced exactly 200 tokens. Nothing about the workload varies across the run. Throughput is the only thing free to move, which is the entire point of the design.

Throughput is not estimated from output length. ollama reports eval_count and eval_duration directly, so each figure is an exact token count over a nanosecond-resolution generation time. Alongside every generation the harness samples macOS thermal state via pmset -g therm, swap in use via sysctl vm.swapusage, and — new since the last run — the power source and battery level, because a first smoke test came in at roughly 50 tok/s on battery against 56 on AC. That difference is large enough to swallow the effect I was trying to measure, so the power source is now stamped on every sample rather than assumed.

This run was on AC. That is the controlled condition, but it is not a neutral one, and I'll come back to why.

What actually happened

The shape is not the cliff the folklore predicts, and it is not a steady slide either. It's three distinct things in sequence.

It decays, but only for about five minutes. The first two minutes average 56.27 tok/s with a spread of ±0.9% — that is the burst figure, and it is the number a benchmark reports. It lands inside the range the previous post on this site measured for this exact model — a median of 54.6 tok/s across five runs, spread 47.9 to 58.0 — which is a useful cross-check that this run started from the same place that one finished. From there it falls, steeply at first: fitting the first ten minutes alone gives −0.591 tok/s per minute.

Then it stops falling. Between minutes 10 and 20 the machine holds 51.73 tok/s across 144 consecutive generations at a spread of ±1.6% — nearly as tight as the cold burst was. This is the finding I care about most. A fanless M1 under continuous inference does not slide indefinitely; it finds a thermally sustainable operating point and sits on it, and that point is only 8.1% below the figure a two-minute benchmark would have told you.

And then something happened at minute 24 that I cannot attribute to heat.

Read that chart for the spread as much as the height. The plateau isn't merely slower than the burst — it's stable, at ±1.6%. The disturbed window is seven times noisier at ±12.2%, and a window that noisy is not the same kind of object as a window that tight. Averaging them together produces a number that describes neither.

The part I can't explain, and won't pretend to

At minute 23.6, swap in use dropped from 585 MB to 427 MB. Within ninety seconds it rose to 891 MB. Across the same window throughput fell to a run-low of 34.4 tok/s and prompt evaluation slowed from around 23 ms to around 32 ms — the machine got slower at everything, not just generation. By minute 28 it had partially recovered to roughly 51 tok/s.

A thermal ceiling does not look like this. Heat accumulates and dissipates on a scale of minutes and moves smoothly; it does not drop 158 MB of swap and then double it inside ninety seconds. What that pattern looks like is a memory-reclaim event or another process waking up on a machine with 8 GB of RAM — which is to say, something I did not control for and cannot identify after the fact.

I could have quietly reported the run mean and nobody would have known. I'd rather write down that the last third of this run contains an event I can't attribute, because the alternative is publishing a thermal finding that is partly somebody else's background process.

My own analysis script was giving me a misleading headline

This is the part worth sitting with. analyse-thermal.mjs, which I wrote and committed yesterday specifically to avoid overstating a weak effect, reports this run as:

throughput      mean 51.13 tok/s   sd 4.33
  first 84      55.57 tok/s   sd 1.40
  last  84      45.28 tok/s   sd 5.15
  change        -18.5%
 
trend           -0.346 tok/s per minute
  R2            0.475  (share of variance the trend explains)
  slope SE      0.018 tok/s per minute
  t             -19.5
  residual sd   3.14 tok/s

Every one of those numbers is arithmetically correct, and the framing is wrong twice over.

The −18.5% compares the first 84 generations against the last 84 — and the last 84 sit squarely inside the disturbance. It is not measuring thermal decay; it is measuring thermal decay plus an unattributed memory event, and attributing the sum to heat.

The linear trend is worse, because it is the more scientific-looking of the two. R² of 0.475 means the line explains under half the variance, and the reason is that the underlying shape is not a line. It is a decay, then a flat plateau, then a disturbance, then a partial recovery. Fitting one slope through four regimes produces a number (−0.346 tok/s per minute) that describes no part of the run: the plateau is flat, and the early decay is nearly twice that steep. Drop the 20–27 minute window and the slope changes to −0.244 with R² 0.526 — a fit that gets better as it explains less of the data, which is the tell.

The commit message for that script says it reports standard error and residual spread "so a weak decay reads as weak instead of being rounded into a headline." It does that honestly and still produced a misleading headline, because I had built it to answer how fast is the decline when the right question was what shape is this. A tool that assumes the answer's form will confirm it. The dataset README now carries this as a documented limitation, and the burst-to-plateau comparison is the figure I'll stand behind.

macOS said nothing, the whole time

In 0 of 424 samples did pmset -g therm report a CPU speed limit, a scheduler limit, or any thermal field at all. The machine measurably slowed by 8% and settled at a lower clock-equivalent throughput, and the operating system's own thermal telemetry registered nothing whatsoever.

I want to be precise about what that does and doesn't establish. It does not prove the settling was thermal — I have no die-temperature reading, because that also requires root. What it does establish is narrower and more useful: pmset silence is not evidence that a machine is running unconstrained. If you are monitoring a fleet of edge devices and using OS-reported thermal pressure as your health signal, this run is a 30-minute counterexample where the signal stayed clean through a real, repeatable, measurable slowdown.

What this means if you run models locally

Quote the sustained number, not the burst. If you have benchmarked a local model over a handful of generations, you have the cold-machine figure. On this hardware the honest steady-state is about 8% lower, and it takes roughly five minutes of continuous load to get there. For capacity planning — how long a document takes to summarise, whether an agent loop finishes inside a coffee break — the plateau is the number that predicts reality.

A fanless machine is better at this than its reputation. I expected a cliff. What I found was a mild settle to a stable operating point, which is a meaningfully different engineering story: passive cooling on this SoC is sized well enough for sustained inference at this model size. That is a real point in favour of small local models on thin hardware, and it is the opposite of what I assumed going in.

On 8 GB, memory is still the thing that bites. The largest single excursion in this run wasn't heat — it was swap. That is the third time measuring this laptop has ended at the same place, and it's consistent with the earlier finding on this site that swap, not GPU memory, sets the real ceiling on what this machine can run.

The repeat run inverted the shape and confirmed the number

I said this post owed a second run before anyone should believe it. Here it is, same machine, same model, same seed and token count, on AC, after a fifteen-minute idle cooldown with nothing else running: 322 generations over 30 minutes.

It did not reproduce the curve. It inverted it.

The second run started slow — 39.5 tok/s against the first run's 56.3 — sagged further to about 30.4 tok/s by minute fifteen, and then climbed steadily to 50.90 tok/s across its final five minutes. Its overall trend is +0.600 tok/s per minute, where the first run's was −0.346.

The cause is legible in the swap column. The second run began with 1,578 MB of swap in use against the first run's 625 MB — the machine had been building this very site minutes earlier — and it spent roughly twenty minutes getting the model's pages resident before it could run properly. On 8 GB, a cold start is not a thermal condition. It is a memory one, and it costs about 40% of throughput until it clears.

And the number I actually care about reproduced. The first run settled at 51.66 tok/s; the second finished at 50.90. That is 1.5% apart, from two runs that arrived at it from opposite directions — one decaying into it, one climbing out of memory pressure to reach it. A figure that survives being approached from both sides is considerably better evidence than the same figure twice.

So, precisely:

• Confirmed. The sustained rate of this machine on this model is about 51 tok/s.
• Not confirmed. The 8.1% burst-to-plateau decay is still one observation. The second run never had a clean burst to decay from, so it tested the destination and not the path.
• Did not recur. The minute-24 collapse appeared in neither the second run's data nor its episode detection — zero sustained episodes. It stays an anomaly, which is exactly what this post said should happen to it until a second run spoke.
• Reproduced. macOS reported thermal pressure in 0 of 322 samples again — 0 of 746 across both runs, through two measurably different slowdowns.

My analysis tool got it wrong a second time

Pointed at the second run, windows.mjs reported a 22.5% decay for a run whose throughput was rising at +0.6 tok/s per minute. It picks the flattest ten-minute window as the plateau, and on a recovering run the flattest window is the slow patch in the middle — the trough it climbed out of.

That is the same failure as the linear fit earlier in this post, in a tool written to avoid it. Both versions assumed a shape and then found it. The tool now reports the stable window and the final window separately, prints the overall trend, and says outright which figure is the sustained rate for a run going in that direction. The lesson I would rather have learned once: an analysis that cannot express the shape it did not expect will confidently mis-describe it.

What I still owe this measurement

Two things, both concrete.

A cold-start repeat. The second run answered the disturbance and the sustained rate, and raised a new question it could not answer: the burst-to-plateau decay needs a run that begins with swap near the first run's 625 MB, which means a genuinely idle machine rather than one that has just finished a build. That is a scheduling problem, not a measurement one.

The energy numbers, still. This run was plugged in, so joules per token remains unmeasured for the same reason as last time: powermetrics is root-only, and on AC the battery gauge reports charging current flowing into the pack rather than system draw. The harness records the power source and emits null instead of a fabricated figure. And there's a wrinkle this run surfaced that I hadn't thought about — the battery charged from 52% to 62% while I was measuring, which means the chassis was dissipating charging heat into the same passive enclosure whose thermal behaviour was the subject. A battery run removes that heat source and finally yields the energy figure. That's the next one.

Reproducing this

scripts/bench/thermal-sustained.mjs drives the run, scripts/bench/analyse-thermal.mjs produces the window statistics, and data/thermal-sustained/README.md documents every field, every window boundary, and every limitation above — including the two ways its own headline figure misleads. All 424 samples from this specific run are committed, so anything here can be recomputed rather than taken on trust.