In March 2021, Google will move to “mobile-only indexing.” For technical SEOs, it’s now critically important that the crawlers they use accurately crawl their sites in a way that truly reflects Googlebot.
That’s why on the 28th January 2021 we made some changes to the way Lumar crawls sites. In particular, for those websites that display mobile versions based on viewport size.
For all sites that serve mobile versions based on user agent, there will be no change to the way Lumar crawls, nor will you see any noticeable differences in the crawl data you see.
For those that serve mobile versions of sites based on viewport size, however, you may notice some changes.
User agent vs viewport size
There are two things you need to keep in mind when crawling a page: user-agent and viewport size.
User agent is a piece of text that identifies the agent requesting the page. That could be Googlebot mobile, Googlebot, Bingbot, or just a Chrome browser. Viewport size, on the other hand, is the pixel size of the screen requesting the resource.
Some websites serve a “mobile version” based on the user agent requesting it. Some websites serve a “mobile version” based on the viewport size. Previously, Lumar used a large desktop viewport size for all crawls.
So for websites that switch to the mobile version of content based on viewport size instead of user agent, we were essentially still crawling the desktop version of the site, but with the mobile user agent.
But Googlebot smartphone user agent also crawls with a smaller viewport, so previously, crawls may not have been a true reflection of what Googlebot sees.
To rectify this, from 28th January Lumar will crawl with a desktop viewport for desktop user agents (1024px wide) and a mobile viewport for mobile user agents (412px wide).
This means that for all websites serving a “mobile version” based on viewport size, all future crawls will be a true reflection of how Googlebot sees your site.
How will this change impact me?
Changes you may see in crawls done with mobile user agents:
A change in LCP and CLS scores (and all other speed scores)
Many speed metrics are impacted by viewport size, so a smaller viewport will change these scores.
A change in total number of URLs crawled.
For sites that change mobile content based on viewport size, the smaller viewport will prompt different content. This content may link to more or fewer URLs, meaning that the total number of URLs crawled may change.
A major change in other metrics.
Many sites have different meta content for mobile and desktop sites. For sites that change meta content based on viewport size, there will be differences in metadata with the new viewport size.
Summary
With “mobile-only indexing” only a few weeks away, it’s important that Lumar accurately crawls your sites in a way that accurately reflects Googlebot – regardless of whether you serve mobile versions using user agent or viewport size. Now, that will be the case.
The changes to the way we crawl will in the coming weeks allow you to make better, more accurate decisions on what changes need to be made to ensure your site isn’t negatively impacted by “mobile-only indexing.”
