Analyzing a slashdotting

On July 22 2003, My website got posted on Slashdot for the second time. This time, the cause of it was my cd changer, which I had added to my website in November of 2002. My previous Slashdot listing, in May of 2002, was caused by my Scanning camera contraption.

I'm very fortunate to have my website on an Excellent ISP who can handle this sort of load, and doesn't give me trouble about occasionally getting lots of hits. Using my shell account, I could also get at the Apache logs. So instead of guesswork or post mortem analysis of a meltdown, I had good data for unimpeded traffic flow.

The person submitting the page to Slashdot did so at 8:01 (the submit itself caused a referral from the submit form, so I could tell).

The next referral came at 8:33. From then until 8:51, I got 26 referrals from Slashdot. At 8:52, things really picked up. This is about when I assume the posting went live for the general public. Slashdot indicates the item was posted at 8:51

I eventually got over 52,000 referrals from Slashdot that day. That's only counting click-thrus where the referring page was indicated as Slashdot, and not counting any subsequent page fetches by users browsing my site afterwards.

There is a noticeable step drop between 0:00 and 1:00 the next morning. This probably corresponds to my page scrolling off the front page of Slashdot. Indeed, there was an article posted at 0:21 on July 23. This item probably pushed the posting off the front page.

Of course, my actual rate of page fetches was substantially higher than my rate of direct Slashdot referrals.

Not surprisingly, my actual hits peaked after Slashdot referrals peaked. This because people who read more of my site probably actually bothered to read the first page, and that takes a bit of time. Plus, people spend hours reading my site (I get e-mails to that effect, and some of the logs indicate this as well).

What is really interesting is how later in the day, I still have way more hits than normal, but they aren't directly from Slashdot. I suspect its that people bookmark the site and read it later, or e-mail it to friends and such. Certainly, at least 95% of the hits that day are due to Slashdot listing. For July 22 and July 23 together, I got over 210,000 page fetches. Normal baseline these days is around 1500 fetches per day, with the occasional spike on the order of about 10,000 per day when the site is referred to from a major blog or Slashdot like page.

Overall, I estimate it took around 10 megabits per second to keep up at peak load. Not astronomical, but way more bandwidth than mere mortals will have on their own web servers.

Go back to my Analyzing traffic logs page.