|
|
|
A Post on the Search Industry Blog
|
|
|
By Nathan Enns on Tuesday, January 17th, 2006 at 12:00 AM
|
Everyone!
They just forget that the simple little file is not only blocking certain robots from specific parts of their website... it is also preventing people from finding potentially useful information and keeping Google from completing its mission of originating the world’s information.
Do you still like the robots.txt file now that you know it is against freedom of information as well as your favorite search engine?
|
|
|
|
