If You Want to Be First on Google Searches, Be Sure to Employ Best Practices for Mobile-First Indexing

On March 26th, 2018, Google announced that after over 18 months of testing they will now begin migrating sites that follow best practices for their mobile-first indexing. This means that Google will now use the mobile version of content to index and rank all websites. Before now, they used the desktop version of a webpage’s content to evaluate how relevant it is to users’ Google searches.

NOTE: You won’t have to change anything if you have a responsive site or a dynamic serving site where the main content and markup is the same in both. However, if your site configurations are different on your mobile and desktop site, you should make some changes to your site. You can check the structured markup across your desktop and mobile versions by typing their URLs into the Structured Data Testing Tool and comparing them.

We’ve been getting a lot of questions about this, so we’re providing some answers for you below.

Why is Google doing this?

Today, most of us search on Google using our mobile devices. Yet, their ranking system is still based on the desktop versions of websites. This creates issues if the mobile pages contain less data than the desktop pages. In this case, the Google algorithms can’t properly evaluate the actual page that’s seen by the mobile device user.

So, to make their results more accurate, Google started experimenting with their index mobile first initiative. And now that it’s been launched, it will be the primary way Google will search for content, show snippets and rank pages.

How does this work?

Google’s Googlebots (or crawlers) search and index web pages. Crawlers are robots or spiders that automatically locate and read websites by “crawling” from one link to another.

Last summer (June 2017) Google advised website owners to switch their m-dot domains to “responsive” before the Mobile-First Indexing launch. If you didn’t do this, then Google will fully index your m-dot content and URLs. This means that the migration for your site will take longer than it should because Google will have to update the content on your pages.

If you have separate desktop and mobile content for your website, this means that you have a dynamic-serving or separate m-dot site specifically designed for mobile devices.

It’s good to have an m-dot site because you can use it to enhance the mobile experience on your site without compromising your desktop or mobile experience.

Ensure you follow Google’s best practices if you want your sites to rank well now with their mobile-first indexing.

Should our mobile content be different than our desktop content? No. It should be the same. And don’t limit it either. This could cause your ranking to decline. If your mobile site contains less content than your desktop site, it’s time to update it. Plus, be sure you include all of your photos, images, and videos in their original indexable and crawlable formats.

What about the structured data? Should we include it on both of our mobile and desktop versions? Yes. The URLs in the structured data on the mobile versions should be updated to the mobile URLs. If you use Data Highlighter for this be sure to check your dashboard regularly for any extraction errors.

How about the metadata? Do we need it on both versions? Definitely. Make sure that titles and meta descriptions are identical across both versions of your site.

Things that you should verify to meet Google’s new Mobile-First Indexing:

• Verify both your mobile and desktop sites in Google’s Search Console. Now that Google has switched over to mobile-first indexing your sites may have experienced a data shift.
• Make sure your mobile site’s hreflang points to the mobile URL and the desktop hreflang points to the desktop URL.
• Make sure your website server has enough capacity for the potential increased crawl rate on your mobile version.
• Use the robots.txt testing tool to be sure that the Googlebot can access your mobile version.
• Be sure that your robots.txt directives work properly on both your mobile and desktop sites.
• Ensure that you have the correct rel=canonical and rel=link elements between your mobile and desktop versions. However, Google says that you shouldn’t have to make changes to your canonical links because they will continue to use them as guides for users who search on either on desktop or mobile sites.
• If you add structured data to your mobile site, don’t add a lot of markups that aren’t relevant to the information.

The Finer Details

• To allow or block Google’s crawlers from accessing any of your content, be sure to specify Googlebot as the user-agent.
• To block pages from Google, blocking the user-agent Googlebot will also block all Google’s other user-agents.
• You can also get more fine-grained control, like having all of your pages show up in Google Search, but not the images in your personal directory. You can use robots.txt to disallow the user-agent Googlebot-image from crawling the files in your personal directory (while allowing Googlebot to crawl all files). Go here for more details on how to do this and for other examples.

Three important points to remember:

1. If you only verified your desktop site in Search Console, you must do so for your mobile version as well.
2.  If you only have a desktop site, Google will continue to index your desktop site but use the mobile agent to do so. This should be fine.
3. If you are still in the process of constructing your mobile version, it’s better to keep using the functional desktop site rather than trusting an incomplete mobile version of the site until it’s ready to launch.

To Summarize:

Google’s ranking, indexing and crawling systems used to use desktop versions of a page’s content. Now, because this may cause problems for mobile searchers they’ll be using the mobile version of a page instead. Google will be showing the mobile version of pages in their Search results and Google cached pages. You should also expect to see increased crawl rates from the Smartphone Googlebot.

You don’t have to worry too much because Google will always present the URL that is most appropriate no matter if it’s a mobile or desktop URL. While their index will be built from mobile documents, they say they will continue to build a great search experience for all users, whether they come from mobile or desktop devices.

If you have any questions, feel free to contact us. Or you can always contact Google via their Webmaster Forum.