Latency numbers every engineer should know
I have been re-reading Tom Limoncelli’s et al. The Practice of Cloud System Administration book (which is great and is well worth reading even 5 years after it has been published) and it has this wonderful reference table called Latency Numbers Every Engineer/Programmer Should Know. This table was popularized by Jeff Dean, and originally presented by Peter Norvig.
I find it handy and wanted to copy it here, on my blog, but then realized it’s almost a decade old as the data was from 2012. This made me do a little research (read: googling) to see how the decade affected the numbers listed. The answer: not much! The only noticeable changes are in disk and network performance thanks to better SSDs/NVMe and 10/100Gb networks.
Which googling though I discovered two interesting bits though: some nice representations of data, and a scaled similar systems latency table from Brendan Gregg’s Systems Performance book. Besides of nice scaling of the latency numbers, that table neatly arranges different subsystems by speed: cpu, memory, disk io, networking, and reboot times.
I have added a scaling columns to the original latency table and updated numbers a bit in both tables to reflect current situations and putting them below for reference.
Latency Numbers Every Engineer Should Know
3GHz CPU cycle 0.3 | 1 s |
L1 cache reference 0.5 | 2 s |
Branch mispredict 3 | 10 s |
L2 cache reference 5 | 20 s | ~10x L1 cache
Mutex lock/unlock ~20 | 1 min |
Main memory reference 100 | 5 min | 20x L2 cache, 200x L1 cache
Compress 1K with Zippy 2,000 | 2 h |
Send 1K over 1Gbps network 10,000 | 10 h | 10Gbps is 10x faster, duh
Read 4K randomly from SSD 150,000 | 6 days | ~1GB/sec SSD
Read 1 MB seq from memory 250,000 | 10 days |
Round trip within same DC 500,000 | 20 days |
Read 1 MB seq from SSD* 1,000,000 | 1 mo | ~1GB/sec SSD, 4x memory
Disk seek 10,000,000 | 10 mo | 20x DC roundtrip
Read 1 MB seq from disk 20,000,000 | 2 yrs | 80x memory, 20x SSD
Send pkt CA->NL->CA 150,000,000 | 12 yrs |
| | | ns| | ~scaled |
| | us|
| ms|
I have updated disk/net numbers with data from this handy site which provides reference historical performance data over the last few decades.
Example Time Scale of System Latencies
3GHz CPU cycle 0.3 ns | 1 s | L1 cache access 0.5 ns | 2 s | L2 cache access 2.8 ns | 9 s | L3 cache access 12.9 ns | 43 s | Main memory access 120 ns | 6 min | Solid-state disk IO 150,000 ns | 6 days | Rotational disk IO 10 ms | 12 mo | Internet: SF to NY 40 ms | 4 yrs | Internet: SF to UK 80 ms | 8 yrs | Internet: SF to AU 185 ms | 19 yrs | TCP packet retransmit 3 s | 317 yrs | Container OS reboot 4 s | 423 yrs | SCSI command time-out 30 s | 3k yrs | VM reboot 40 s | 4k yrs | Physical system reboot 5 min | 32k yrs |
References: