Google indexing .txt weather files

Worf · Post by **Worf** » Wed 10 Jul 2013 10:11 am

Very strange. Google has started indexing all .txt files relating to my weather station. If you search for "www.finghall.com" (as typed) you will see what I mean. Is there a way of adding something to the template for those pages to stop robots crawling them? Thanks.

Post by **steve** » Wed 10 Jul 2013 10:17 am

Use a robots.txt file - http://www.robotstxt.org/

Worf · Post by **Worf** » Wed 10 Jul 2013 10:30 am

Many thanks for that. I realise I can block individual txt files, but which template file do I need to edit in Cumulus to ensure that all newly generated files contain the "no robots" info?

Post by **steve** » Wed 10 Jul 2013 10:33 am

You don't put it in the files themselves, you put it in the robots.txt file.

Worf · Post by **Worf** » Wed 10 Jul 2013 11:23 am

Understood. Just me being thick!

Many thanks

Bruce

Cumulus Support

Google indexing .txt weather files

Google indexing .txt weather files

Re: Google indexing .txt weather files

Re: Google indexing .txt weather files

Re: Google indexing .txt weather files

Re: Google indexing .txt weather files