Latency and Throughput in WWW

The World Wide Wait is the bane of surfers the world over because of latency and throughput.

Before getting into pings, bings, and traceroutes in Web Performance Tuning, Patrick Killelea is kind enough to describe latency and throughput better than I could ever hope to. Here are what he refers to as the three traditional examples:

1. An overnight (24-hour) shipment of 1,000 different CDs holding 500 megabytes each has terrific throughput but lousy latency. The throughput is (500 × 2,201.05 × 8 × 1000) bits/(24 × 60 × 60) seconds = about 49 million bits/second, which is better than a T3's 45 million bits/second. The difference is that the overnight shipment bits are delayed for a day and then arrive all at once, but T3 bits begin to arrive immediately, so the T3 has much better latency, even though both methods have approximately the same throughput when considered over the interval of a day. We say that the overnight shipment is bursty traffic.

2. Supermarkets would like to achieve maximum throughput per checkout clerk because they can then get by with fewer of them. One way for them to do this is to increase your latency, that is, to make you wait in line, at least up to the limit of your tolerance. In his book Configuration and Capacity Planning for Solaris Servers (Prentice-Hall), Brian Wong phrased this dilemma well by saying that throughput is a measure of organizational productivity while latency is a measure of individual productivity. The supermarket may not want to waste your individual time, but it is even more interested in maximizing its own organizational productivity.

3. One woman has a throughput of one baby per 9 months, barring twins or triplets, etc. Nine women may be able to bear 9 babies in 9 months, giving the group a throughput of 1 baby per month, even though the latency cannot be decreased (i.e., even 9 women cannot produce 1 baby in 1 month). This mildly offensive but unforgettable example is from The Mythical Man-Month, by Frederick P. Brooks (Addison Wesley).

Starting again from the customer's perspective (I do hope you're beginning to see a pattern here), the first slowness is experienced while waiting for your home page to appear at all. So what's the holdup? Assuming everything is hunky-dory on your end, the problem may lie in how your bits get out to the rest of the world.

A number of vendors have come forth to monitor the Internet stopwatch, vendors like Keynote Systems (www.keynote.com). On its site, Keynote shows off the speeds of the faster well-known sites on the Web. Yahoo! and FedEx are neck and neck in the subsecond home page response race, according to Keynote Systems.

Keynote measures your Web site from afar to make sure it's working to your liking. They will test your home page, your login pages, your email servers, your pagers, and then some. They will test all of the above from (last time I checked) as many as 106 locations/networks around the world. Keynote has a number of ways of looking at your site. They measure the time it takes to grab and download a single picture or a whole page. They compare how long it takes to execute a multipage, interactive transaction. They monitor the quality of streaming content. They measure the time it takes to access and download pages over a 56-kilobyte modem, a DSL connection, or a cable connection.

If there's a problem, Keynote will email you a notification or page you. But, of course, it offers more than just throwing a red flag in your face. If you've got problems, Keynote has consultants who can help even if the problem isn't on your server. In the email Keynote sends when you first sign up, it assures you that you're not always to blame:

Please note that over 75% of all instances of site inaccessibility detected by Keynote Red Alert are NOT due to problems with the server, software, LAN, or ISP connection (though Keynote Red Alert detects these problems also). The vast majority of instances of site inaccessibility are due to problems with your "extended global connectivity", that is, problems with the Internet backbones to which you connect, their peering with other backbones, and the routes that are broadcast to direct traffic to your site.

Expect to be surprised about problems you did not know existed. Also, understand that there IS something you can do about it, and that our staff can assist you in working with your bandwidth providers to zero in on and resolve these problems. Our customers include the world's largest ISPs and Internet backbone providers, because they know that monitoring your site from within its own network simply WILL NOT detect "extended connectivity" problems, which are THREE TIMES more prevalent than LAN and server problems.

So what do you do if access to your site from different parts of the world is so slow it's scaring away business? You either duplicate your equipment and data in strategic places so that everybody has access to a more local server or you follow Oracle's lead.

Rene Bonvanie, vice president of Online Marketing at Oracle, worries about latency and throughput. He has to-all of his eggs are in one basket. "Whether you're in South Africa or China or the U.S.," he says, "everything comes from a single server." How do Web pages get to the other side of the world fast enough? "We have agreements with bandwidth providers." Oracle's approach is to pay for the bandwidth from here to there, wherever there might be. If your site simply refuses to show up at all, it's not a latency problem or a throughput problem. Instead, you're going to be kept awake at night by availability anxiety.