What a very large audience taught me about web performance

The Horseshoe Falls at Niagara from the air, a wall of white water curving into a churning green river

Building web software for a very large audience changed what I believe about performance. Not by teaching me new techniques, but by showing me how wrong my instincts were when the audience was small enough that everyone looked like me.

The average is a lie

With millions of users an average load time means almost nothing, because the distribution is so wide the average describes nobody. A median of two seconds can hide a tenth of your users waiting fifteen, and that tenth is not random. It is the people on old phones, slow networks, and far-away connections, with the fewest alternatives if your site does not work.

I learned to look at the 90th and 95th percentiles first and the median last. I think a team should pick the percentile it is willing to defend and put that number on the wall, because the average will always look fine and the people in the tail will leave quietly.

The device on your desk is not the audience

Engineers build on fast machines, over fast connections, in the newest browser. A page that feels instant on a development laptop can be unusable on the mid-range phone most of the world uses, and the engineer never sees it.

At scale, I stopped trusting any performance judgment made on a developer machine. The team needs real devices, the kind the audience has, and field data from real users, because the lab is always kinder than the world.

Bytes are the only honest budget

There are many performance metrics and most can be argued about. Bytes cannot. A megabyte of JavaScript is a megabyte on every device, every time, and it costs the most on the devices that can least afford it.

So I think a bytes budget, per page, enforced in the build, is the most useful performance rule a large-audience team can adopt. Not because bytes are the whole story, but because they are the part nobody can rationalize away.

The long tail is the audience

Much of a very large audience is unusual. The user on a screen reader, the user whose browser is three versions old. Each group is a small percentage. Together they are a large share of everyone, and the share most likely to be hurt by an assumption.

At scale, I stopped thinking of the long tail as edge cases and started treating it as the audience, with the fast-connection, new-phone user as the edge case.

Performance needs an owner

Performance regresses by default, one reasonable change at a time. The only teams I have seen hold the line are ones where someone owns it, watches the field numbers, runs the budget, and has the standing to say no to a feature that costs more than it is worth.

I think that owner is a role, not a task, and should be staffed the way security is. The audience will not file a bug. It will just be smaller next quarter.

Photo source: https://photos.robertstowe.com/niagara