Can you trust Alexa?
Alexa is a California-based subsidiary company of Amazon that is best known for operating a website that provides information on the web traffic to other websites. Alexa collects information from users who have installed an "Alexa Toolbar," allowing them to provide statistics on web site traffic, as well as lists of related links.
Alexa ranks sites based on visits from users of its Alexa Toolbar for Internet Explorer and from integrated sidebars in Mozilla and Netscape.
There is some controversy over how representative Alexa's user base is of typical Internet behavior. If Alexa's user base is a fair statistical sample of the internet user population (e.g., a random sample of sufficient size), Alexa's ranking should be quite accurate. In reality, not much is known about the sample and possible sampling biases.
Another concern is whether Alexa ratings are easily manipulated. Some webmasters claim that they can significantly improve the Alexa ranking of less popular sites by making them the default page, by exchanging web traffic with other webmasters, and by requiring their users to install the Alexa toolbar; however, such claims are often anecdotal and are offered without statistics or other evidence.
It is possible for a site's three-month traffic rank to be higher than any single daily rank shown in the Trend graph. On any given day there may be many sites that temporarily shoot up in the rankings. But if a site has consistent traffic performance, it may end up with the best ranking when the traffic data are aggregated into the three-month average. A good analogy is a four-day golf tournament: if a different player comes in first at each match, but you come in second at all four matches, you can end up winning the tournament.
The bottom line is Alexa rankings should be used to provide relative and not explicit rankings of a site. I use Alexa all the time to do trend analysis on target and competitive sites. You get a fair idea of whether the sites momentum is up, down or static. That is generally enough for me to decide whether to proceed with a more detailed investigation or to put an opportunity on the back burner.
NOTE: Significant sections of this post were referenced from Wikipedia and are used under their GNU Free Documentation License
Scouta was recently featured at Techcrunch and several weeks later Boing Boing. The later drove far more traffic--about three times as much on one single day--and yet the Alexa graph didn't record much more than an average days page views.
Perhaps with the trend to move to Firefox as the browser of choice, many people aren't installing the appropriate plugin.
I'd say Alexa is only useful to analyze the trend of the very top sites.