Indexing
In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.
Use ‘Send Feedback’ Link to Alert Google to Issues in the SERPs
You can use the ‘Send Feedback’ link at the bottom of the search results to alert Google to issues such as multiple results from one site with similar content being returned.
Check Log Files to See Switch to Mobile-first Indexing
Webmasters will spot the switch to Mobile-first indexing in log files. Currently, ~80% of crawling is done with Googlebot Desktop and ~20% with Googlebot Smartphone. You will likely see this shift over, with more crawling being done with Googlebot Smartphone.
Drop in Indexed Pages Could be Google Removing Duplicates
Google recognises duplicate pages within the GSC index status report. As URLs are reprocessed over time, Google realises that they don’t need to be kept so this can be one reason behind a drop in indexed pages and is commonly seen following HTTPS migrations.
Pages That Redirect to Interstitials Are Dropped from the Index
Make sure pages don’t redirect to interstitials and that they’re kept on the same URL or Googlebot could drop these pages from the index.
Use Several Smaller Sitemaps to Locate Indexing Issues
Having several smaller sitemaps for the different sections of your site is recommended for diagnosing indexing issues.
Site:Query Isn’t Indicative of Total Number of Indexed URLs
The number of URLs returned using a site: query is optimised for speed so isn’t indicative of the total number of indexed URLs. It is better to use Search Console’s sitemap section, which shows you how many URLs were indexed of those submitted in the sitemap file.
High Quality Sites Are Prioritised When Using Submit To Index Feature
Higher quality sites can be given preference over low quality sites when using the Submit to Index feature.
URLs in Sitemaps Are Not Guaranteed to be Indexed
Google may choose not to index URLs in sitemaps that are very similar to ones already being indexed and if they differ to the ones linked within the site (e.g. trailing slash/non-trailing slash).
Google Isn’t Always Able to Show Cached Versions of a Page
Google is unable to show the cached version of a page if it has a noarchive tag, or any JavaScript based content which isn’t included in the cache.
Google Uses Multiple Signals to Choose HTTP or HTTPS URLs
Redirects, internal links, sitemaps, rel canonicals are taken into consideration when Google chooses to index a page on HTTP or HTTPS. Internal links to HTTP URLs after a migtration to HTTPS will give Google conflicting signals, but usually there will be enough signals to indicate that the HTTPS version should be indexed.