What a very large audience taught me about web performance

The Horseshoe Falls at Niagara from the air, a wall of white water curving into a churning green river

Building web software for a very large audience changed what I believe about performance. Not by teaching me new techniques, which are well documented, but by showing me how wrong my instincts were when the audience was small enough that everyone looked like me.

The average is a lie

With a small audience, an average load time means something. With millions of users it means almost nothing, because the distribution is so wide that the average describes nobody. A median of two seconds can hide a tenth of your users waiting fifteen, and that tenth is not random. It is the people on old phones, slow networks, and far-away connections, which is to say the people with the fewest alternatives if your site does not work.

I learned to look at the 90th and 95th percentiles first and the median last. I think a team should pick the percentile it is willing to defend and put that number on the wall, because the average will always look fine, and the people in the tail will always leave quietly.

The device on your desk is not the audience

Engineers build on fast machines, over fast connections, in the newest browser. Every one of those is a thumb on the scale. A page that feels instant on a development laptop can be unusable on the mid-range phone that most of the world actually uses, and the engineer will never know, because nothing in their day shows them.

At scale, I stopped trusting any performance judgment made on a developer machine. The team needs real devices, the kind the audience actually has, and a habit of using them. It also needs field data, measurements from real users' browsers, because the lab will always be kinder than the world.

Bytes are the only honest budget

There are many performance metrics and most of them can be argued about. Bytes cannot. Every byte you ship has to cross a network you do not control, be parsed by a processor you did not choose, and sit in memory you do not own. A megabyte of JavaScript is a megabyte on every device, every time, and it costs the most on exactly the devices that can least afford it.

So I think a bytes budget, per page, enforced in the build, is the single most useful performance rule a large-audience team can adopt. Not because bytes are the whole story, but because they are the part of the story nobody can rationalize away, and a team that holds the line on bytes ends up holding the line on most other things too.

The long tail is the audience

The surprising thing about a very large audience is how much of it is unusual. The user on a screen reader, the user in a country where your fonts are not cached, the user whose browser is three versions old because their employer locks it. Each group is a small percentage. Together they are a large share of everyone, and they are the share most likely to be hurt by an assumption.

At scale, I stopped thinking of the long tail as edge cases and started thinking of it as the audience, with the fast-connection, new-phone user as the actual edge case, because that was closer to the truth.

Performance needs an owner

The last lesson is organizational. Performance regresses by default, one reasonable change at a time, and nobody notices because each change is small. The only teams I have seen hold the line are ones where someone owns it: watches the field numbers, runs the budget, and has the standing to say no to a feature that costs more than it is worth.

I think that owner is a role, not a task, and a team building for a large audience should staff it the way it staffs security. The audience will not file a bug. It will just be smaller next quarter.

Photo source: https://photos.robertstowe.com/niagara