2014/10/16 上午 12:07:14

jquery Programming Glossary: robots.txt

404 errors on /a folders in Webmaster Tools

http://stackoverflow.com/questions/5416081/404-errors-on-a-folders-in-webmaster-tools

jQuery itself is this line... a href ' a' style 'color red float left opacity .55 ' a a EDIT 4 Block it with the robots.txt file. According to Google this will only change the Webmaster Tools Crawl Error from a 404 to a Blocked by robots.txt Disallow.. robots.txt file. According to Google this will only change the Webmaster Tools Crawl Error from a 404 to a Blocked by robots.txt Disallow a Disallow a EDIT 5 The following is a response from Google employee JohnMu on this issue in the thread I started.. to your homepage disallowing the URL will also bring it up as a crawl error it will be listed as a URL disallowed by robots.txt . I would also recommend not explicitly disallowing crawling of the jQuery file. While we generally wouldn't index it on..

jQuery causing 404 errors in Webmaster Tools on /a directory

http://stackoverflow.com/questions/5749348/jquery-causing-404-errors-in-webmaster-tools-on-a-directory

Does anyone know why how the Googlebot would go inside jQuery EDIT I've since blocked the jQuery file with the robots.txt file but I really wasn't expecting the Googlebot to go into external JavaScript files. EDIT 2 The following is a response.. to your homepage disallowing the URL will also bring it up as a crawl error it will be listed as a URL disallowed by robots.txt . I would also recommend not explicitly disallowing crawling of the jQuery file. While we generally wouldn't index it on.. include but maybe a bot would decide to parse. In any event rather than setting a redirect why don't you just use robots.txt Add this line Disallow a You could also try fixing jQuery. Obfuscating the link a little bit would probably do the trick..