Official Google answer: Does Googlebot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it?

Question:

Does Googlebot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it? Or does it stick entirely to what it finds via the link graph and/or Sitemaps/feeds?

Google's Matt Cutts answers:

Summary of the answer:

The summary of the answer is available to paid members. Become a member now. It's fast, easy and risk-free.

Question	Views
Regarding "nofollow" on internal links: Does it hurt?	459
If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using Meta Robots noindex,nofollow a better alternative?	350
How would Google consider (and rank) a site that uses meta data and URLs in a language (Italian) and has the H1 of the pages in another (English) considered more appealing for users?	323
Tips on requesting reconsideration	494
Links from relevant and important sites have always been a great way to get traffic and acceptance for a website. How do you rate links from new platforms like Twitter, Facebook to a website?	549
Websites lose backlinks due to other websites going out of business or closing (Geocities, AOL member pages). Does Google remove the back link juice that once came from these pages?	356
If I link back to a press article about our company, will Google perceive this as a reciprocal link and therefore give less weight to the inbound link?	293
How does Google calculate site load times in the data it exposes in Google's webmaster statistics?	232
We are a pretty big site. We are changing our hosting company in the next few weeks (same country). Should we be scared from an SEO perspective?	414
How can Googlebot crawl and index pages that don't have any links to them on my website?	394

Question

Views

Regarding "nofollow" on internal links: Does it hurt?

459

If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using Meta Robots noindex,nofollow a better alternative?

350

How would Google consider (and rank) a site that uses meta data and URLs in a language (Italian) and has the H1 of the pages in another (English) considered more appealing for users?

323

Tips on requesting reconsideration

494

Links from relevant and important sites have always been a great way to get traffic and acceptance for a website. How do you rate links from new platforms like Twitter, Facebook to a website?

549