Where Has The Performance Gone?

My CPU got more than ten-thousand times faster over the last 30 years. What have we done with this astonishing performance gain?

Sure, we got nicer displays, 4k video playback, real-time 3D graphics, and 10-gigabit USB in the much smaller form-factor of a laptop. These feature were definitively out of reach in 1993.

However, our machines do not feel that much faster. So where is the gap coming from between what seems to be possible and what is actually available to the user?

The best explanation I found is that the mentioned performance increase only looks at the peak throughput of the system but completely ignores any latency. But latency is what improved a lot less over the years.

Consider for instance a system-call into the Linux kernel. This operation took at least two microseconds in 1995. On my current machine we are down to 50 nanoseconds. That is only a 40x boost and perfectly linear to the increase in CPU frequency.

It looks like we have to focus on latency again to make our systems more responsive. What do you think?