[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

yahoo crawlers hammering us



> Possibly because that other user is who the customer pays have their content
> delivered to?

Customers don't want to deliver their content to search engines?  That seems silly.
 
http://www.last.fm/robots.txt (Note the final 3 disallow lines...)