September 22, 2020

Latency numbers every engineer should know

latency numbers

I have been re-reading Tom Limoncelli’s et al. The Practice of Cloud System Administration book (which is great and is well worth reading even 5 years after it has been published) and it has this wonderful reference table called Latency Numbers Every Engineer/Programmer Should Know. This table was popularized by Jeff Dean, and originally presented by Peter Norvig.

I find it handy and wanted to copy it here, on my blog, but then realized it’s almost a decade old as the data was from 2012. This made me do a little research (read: googling) to see how the decade affected the numbers listed. The answer: not much! The only noticeable changes are in disk and network performance thanks to better SSDs/NVMe and 10/100Gb networks.

Which googling though I discovered two interesting bits though: some nice representations of data, and a scaled similar systems latency table from Brendan Gregg’s Systems Performance book. Besides of nice scaling of the latency numbers, that table neatly arranges different subsystems by speed: cpu, memory, disk io, networking, and reboot times.

I have added a scaling columns to the original latency table and updated numbers a bit in both tables to reflect current situations and putting them below for reference.

Latency Numbers Every Engineer Should Know

3GHz CPU cycle                     0.3 |   1 s    |
L1 cache reference                 0.5 |   2 s    |    
Branch mispredict                  3   |  10 s    |
L2 cache reference                 5   |  20 s    | ~10x L1 cache
Mutex lock/unlock                ~20   |   1 min  |
Main memory reference            100   |   5 min  | 20x L2 cache, 200x L1 cache
Compress 1K with Zippy         2,000   |   2 h    |
Send 1K over 1Gbps network    10,000   |  10 h    | 10Gbps is 10x faster, duh
Read 4K randomly from SSD    150,000   |   6 days | ~1GB/sec SSD
Read 1 MB seq from memory    250,000   |  10 days |
Round trip within same DC    500,000   |  20 days |
Read 1 MB seq from SSD*    1,000,000   |   1 mo   | ~1GB/sec SSD, 4x memory
Disk seek                 10,000,000   |  10 mo   | 20x DC roundtrip
Read 1 MB seq from disk   20,000,000   |   2 yrs  | 80x memory, 20x SSD
Send pkt CA->NL->CA      150,000,000   |  12 yrs  |
                        |   |   | ns|  | ~scaled  |
                        |   | us|
                        | ms|

I have updated disk/net numbers with data from this handy site which provides reference historical performance data over the last few decades.

Example Time Scale of System Latencies

3GHz CPU cycle                  0.3 ns  |   1 s    |
L1 cache access                 0.5 ns  |   2 s    |    
L2 cache access                 2.8 ns  |   9 s    |    
L3 cache access                12.9 ns  |  43 s    |    
Main memory access            120   ns  |   6 min  | 
Solid-state disk IO       150,000   ns  |   6 days |
Rotational disk IO             10   ms  |  12 mo   |
Internet: SF to NY             40   ms  |   4 yrs  |
Internet: SF to UK             80   ms  |   8 yrs  |
Internet: SF to AU            185   ms  |  19 yrs  |
TCP packet retransmit           3   s   | 317 yrs  |
Container OS reboot             4   s   | 423 yrs  |
SCSI command time-out          30   s   |  3k yrs  |
VM reboot                      40   s   |  4k yrs  |
Physical system reboot          5   min | 32k yrs  |

References:


— `If you knew Time as well as I do,' said the Hatter, `you wouldn't talk about wasting IT. It's HIM.'
$ Last updated: Oct 17, 2020 at 18:09 (EEST) $