Indexing
In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.
PubSubHubbub is the fastest way to get content into Google
RSS feeds with PubSubHubbub are a quickest way to get content updated in Google.
301 and 302 Redirects Only Determine Which URL is Indexed
A 301 indicates to Google that the destination URL should be indexed, a 302 indicates to Google that the original URL should be indexed but they always use the content on the destination page.
Google Filters Identical Duplicates During indexing, and Near Duplicates From Search Results Pages
When Google recognises identical pages, it will choose one version to index, and when pages are similar, only one may show up in search results. Google looks at factors such as rel canonicals, redirects and internal and external linking when identical pages are crawled to decide which one to index.
Near Identical Pages with HREFLANG may be Rolled Together
If you have identical pages which only differ a very small amount, such as a currency, Google may roll the pages together, but use HREFLANG to decide which one to show in search results.
Google Chooses the HTTPS Version of a Page if Both Exist
Google will choose the HTTPS version of a page instead of HTTP when both exist, but there is no change in ranking position.
Contact Google for Sites Incorrectly Filtered by Safe Search
If your website is incorrectly being filtered by safe search, you can submit a form in the help centre to ask Google to reclasify it.
URL Parameters Help Crawling and Indexing
URL parameters in URLs make it easier for Google to understand URLs for crawling and indexing. If you put everything into the path of the URL it can be harder for Google to crawl them properly.
Disallowed URLs may Show in Search Due to Internal Linking
If you’re seeing disallowed URLs showing up in search results, it may be because of internal linking to these pages, so you should change links to point to pages you want to be shown.
Sitemap Index Counts Report the Exact Submitted URLs
Sitemap Index counts report the exact URL you sumbit including trailing slashes. If Google chooses to index a different copy of the same page, the submitted URL wouldn’t be reported as indexed.
Cross Linking Deep Pages Can Help Indexing
Cross linking between deep pages linked through paginated category pages can help them be discovered more easily.