Another source of distortion affecting the information provided by systems for web log analysis concerns the visits coming from robots (crawlers, spiders, bots and the like).

DAEDALUS verifies daily in many web sites that, depending on its age, size, content volatility, etc. more than 50% of the visits received by a web site can be generated by robots, specially Internet search engines (e.g., the robots from Google, LookSmart, Inktomi, Fast, AskJeeves, Altavista, etc.)

Log-based analyzers do not distinguish, in general, between visits coming from humans and visits from artificial agents, not being able of filtering access from robots. As a consequence, the statistical information that they deliver can be useful for webmasters (giving an idea of bandwidth consumption), but is useless for the marketing department or for the business management. Follow the link to the next page to know better solutions to this problem.