Get help and swap ideas with other webmasters and business owners

248 official Google answers to common SEO questions

Google's Matt Cutts answers SEO questions

Matt Cutts works for the search quality group in Google, specializing in search engine optimization issues. He is well known in the SEO community for enforcing the Google Webmaster Guidelines and cracking down on link spam.

Search our Q & A database to find official answers to your SEO questions. Watch the videos in which Matt Cutts answers questions or read short summaries of the answers.

Get answers
Enter your question:

All questions about the topic "Crawling":

Question Views
1. Links from relevant and important sites have always been a great way to get traffic and acceptance for a website. How do you rate links from new platforms like Twitter, Facebook to a website? 544
2. Uncrawled URLs in search results 529
3. Should I disallow Googlebot from crawling slower pages? 510
4. How can I make sure that Google reaches and indexes pages that are on a lower (deeper) level of a website? 506
5. Will Google consider Yahoo! Directory and BOTW as sources of paid links? 500
6. HTML sitemap vs. XML sitemap. Which one is yummy for Google search engine spider? 471
7. As Google's algo evolves, is it better to have exceptional links and mediocre content, or exceptional content and mediocre links? 467
8. Is first link priority an on-page SEO factor? 465
9. How effective is Google now at handling content supplied via Ajax? 459
10. How much time is Google taking to index a new webpage, and how can we accelerate the process besides using Google Webmaster Tools? 457
11. Regarding "nofollow" on internal links: Does it hurt? 453
12. What are your views on 'PageRank sculpting'? 453
13. PHP performance tips 444
14. What impact do site load times have on Google rankings? 441
15. Are there any APIs available from Google to pullout reports from Google Webmaster Tools? 435
16. Why does Google crawl/index blogs (specifically sites notified by "WordPress XMLRPC pings") so much faster than a "normal" site submitting a revised Sitemap. What is the impact of that on the overall "quality" of the index? 430
17. Is there a limit to the number of pages that Google will index from one site? 430
18. How does Google determine domain age, and is it important for ranking? 427
19. Is a website designed with a CSS-based layout more SEO friendly than a table-based layout? 415
20. We are a pretty big site. We are changing our hosting company in the next few weeks (same country). Should we be scared from an SEO perspective? 408
21. How active is WMT (Webmaster Tools) monitored by Google, specifically when there is a system error (such as the recent error in reporting the number of pages indexed)? 403
22. Now that Google can crawl JavaScript links, what is going to happen with all those paid links that were behind JavaScript code? 402
23. Is it a good thing to put 'nofollow' in links to a disclaimer, privacy statement and other pages like that with the internal PageRank in mind? 400
24. Does Google have any suggestions (or data) on the impact of pipes versus dashes in the title tag? 398
25. The Sitemap.xml file states there are 10,000 URLs but only 1500 have been indexed. After numerous crawls it does not appear Google is going to index these additional detail pages. What can I do to get Google to index my unique and current detail pages. 393
26. In regards to the new canonicalization tag, does it make sense for large corporations to consider placing that tag on every page due to marketing tracking codes and large levels of duplicate URLs like faceted pages and load balancing servers? 391
27. Specifying an image's license using RDFa 389
28. How can Googlebot crawl and index pages that don't have any links to them on my website? 389
29. Can Google provide a way to mark a section of our pages as being less important for being indexed/snippeted by Google? 386
30. Which search media returns more reliable information: Google or Twitter? 386
31. Does Google crawl and treat TinyURLs using a 301 redirect the same as other links? 378
32. How does Google handle ligatures, soft-hyphens, interpuncts and hyphenation points? 371
33. Do dates in the URL of blogs or websites help determine freshness of the content or is it largely ignored? 364
34. I am using a template website (I'm an amateur!). The H1 tag appears below the H2 tag in the code. Does the spider know what's going on? 362
35. How not to hide text 362
36. Say your index page has been cached by Google and then you change the meta description. How long does it take for a Google bot to recrawl that page? 360
37. What are Google's plans for indexing the deep web? 357
38. Websites lose backlinks due to other websites going out of business or closing (Geocities, AOL member pages). Does Google remove the back link juice that once came from these pages? 350
39. How much does the size of a web site (# indexed pages/content) have an effect on its authority in Google's eyes? 350
40. AdWords keyword tool gives an estimate of search traffic for a specific (or broad) keyword - How much (%) of this traffic do you believe are search marketers, SEOs, analysts and even business owners etc searching their own targeted keywords? 346
41. Google announced page load speed matters for ranking. Should we be doing content-only pages for Google bots? 344
42. If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using Meta Robots noindex,nofollow a better alternative? 343
43. Should a "Sale Page" be in a robots.txt file to avoid duplicate content? 341
44. Optimizing the order of scripts and styles 340
45. How reliable is the 'site:domain.com' query in determining the number of pages in the Google index? 334
46. We still have old content in the index. We block them via robots.txt, use 404 and delete via Webmaster Tools, but Google still keeps it. What can we do to quickly delete content from the index? 333
47. How does Google rank sites which run on a different port than the standard port 80? 320
48. Minimizing browser flow 317
49. How would Google consider (and rank) a site that uses meta data and URLs in a language (Italian) and has the H1 of the pages in another (English) considered more appealing for users? 317
50. What is the best way to serve different content according to user country IP (legal reasons)? 310
51. Does using a class or an id in a header tag: <h1 id="whatever">text</h1> instead of plain headers: <h1>text</h1> interfere with the way search engines see and understand headings? 310
52. Last year, one of my client's web servers when down for over a day. Would this have affected the site's PageRank at all? 304
53. If a page is disallowed by robots.txt, will a link to this page transfer/leak link juice? 304
54. Can moving my website to 'the cloud' harm my listings? 302
55. A question to non-intended duplicate content: If an online shop can be reached through several TLDs (like .de, .at, .ch) and the only difference is the currency (and necessarily the checkout process) does Google consider this duplicate content? 298
56. If we were to syndicate my written content (entire articles) to multiple domains then would we be able to use the imminent cross-domain <link rel="canonical" tag to confirm which site we would like to index for a given piece of content? 297
57. What is the benefit of using the Change of Address tool in Google Webmaster Tools, compared to just setting up the required 301 redirections to the new site? 293
58. An orphanage website I work on is showing up for searches on "girls in bathrooms" because they have an article about renovating the girls' bathroom! What do you think of the idea of a negative keyword meta tag to block irrelevant searches? 293
59. Is it possible to exclude Experts Exchange from search results? 288
60. How will Google search work with dynamic HTML pages (and I don't mean JSP or other Web 1.0 technologies), like applications that are built with GWT? 279
61. How many bots/spiders does Google currently have crawling the web? 278
62. Does Googlebot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it? 276
63. Are you ever going to do 'weather reports' like Yahoo! does algorithm updates? 274
64. Does PageRank take into account cross-browser compatibility? 274
65. Will DiggBar create duplicate content issues? 272
66. What is the nofollow equivalent for JavaScript links/redirections (now that you follow those too)? 270
67. If I externalize all CSS style definitions and JavaScript scripts and disallow all user agents from accessing these external files (via robots.txt), would this cause problems for Googlebot? 269
68. What is the best way to deal with BIG sitemaps.xml (e.g. more than 1,000,000 pages)? 268
69. I noticed that, for example, "Texas widget", and "widget Texas" return different results. I think the gist is the same but the results were different. I'd like to include both terms/phrases on my page but wouldn't that be considered keyword spamming? 264
70. Can I use robots.txt to optimize Googlebot's crawl? 260
71. I have a server-side script that automatically redirects visitors to a mobile version of a site if they are using a mobile browser. My question is: What are some things to watch out for (if any) when serving different content based on the visitor? 256
72. Any reason why Google search does not treat the @ symbol differently given the rise of Twitter? 254
73. Are Chrome's 'usage statistics' used in evaluating site speed? 252
74. I hate IE6! How would you propose we rid the internet of this outdated browser? 248
75. Is there a way to tell Google bots to exclude recurring words on a website such as "leave comment" or "print page" when indexing in order to improve the keyword density? 247
76. Can we feed Googlebot a version of a page that does not contain any advertising code (JavaScript or otherwise)? 244
77. We work on a well established website. Mobile web seems to becoming more and more popular - should we create a mobile version of this site? 243
78. How does Google calculate site load times in the data it exposes in Google's webmaster statistics? 228
79. Following your interview with Eric Engel - you mention about "If Modified-Since." We worked on many websites whereby the actual file timestamp doesn't change but the content does as the pages are database-driven. How should we deal with such situations. 227
80. Is there a good way to kick off a feed in Google Reader by doing something like temporarily making the feed include a whole bunch of old content? 226
81. Will the new canonical tag help with issues where you, by accident (stupid editors linking to wrong addresses) have indexed sites by the IP address rather than hostname? 219
82. "Real-time indexation" on Google, when we use site:www.sitename.com; is this a possibility in the near future? 210
83. On a web retail site, unique item descriptions are ideal for both users and Googlebot, compared to generic manufacturer descriptions. Some users prefer to see generic descriptions, too. Will including both reduce significance of the unique content? 201

The Google Q & A database is available to paid members

Get full access to all tools now!

Become a member now and access all SEOprofiler tools without limitations. It's fast, easy and risk-free. Test it and see for yourself!