 |
|
|
Thread Tools | Display Modes |
02-08-2006, 10:10 AM
|
||||
|
||||
robots.txt
Good info from Matt Cutts
Despite good info on the web, even experts can have a hard time knowing with 100% confidence what a certain robots.txt will do. When Danny Sullivan recently asked a question about prefixing matching, I had to go ask the crawl team to be completely sure. Part of the problem is that mucking around with robots.txt files is pretty rare; once you get it right once, you usually never have to think about the file again. Another issue is that if you get the file wrong, it can have a large impact on your site, so most people don’t mess with their robots.txt file very often. Finally, each search engine has slightly different extra options that they support. For example, Google permits wildcards (*) and the “Allow:” directive. http://www.mattcutts.com/blog/
__________________
1stPageSEO.com . Reverse Mortgage Lenders . Lake James Waterfront Home Contact us for fast, efficient and cost-effective web design, content creation, search engine optimization, blog design and maintenance. 
|
|
«
Previous Thread
|
Next Thread
»
| Thread Tools | |
| Display Modes | |
Linear Mode
|
|
