Google’s Indexing Glitch: What Happened on August 10th & Why Indexing Bugs Occur

On August 10, 2020, Google's index went a little haywire.

Many SEOs began to notice pretty substantial fluctuations in their rankings that, frankly, didn't make a whole lot of sense.

Recipe results are a disaster. Half the SERPs aren't even returning the search query. I really hope this gets fixed ASAP! A search for "paleo chocolate cake" is returning chicken, oatmeal and ice cream! pic.twitter.com/5ckp1Vhx7d

-- Lisa Bryan (@Downshiftology) August 11, 2020

While some attributed the volatility to an algorithm update early on, others assumed that the strangeness they were seeing on the SERP must have been the result of a bug.

Turns out, they were right.

Google confirms that it was an indexing bug, rather than an algorithm update, that caused the volatility

A few hours after the chatter began, Google's John Mueller commented that, although he didn't yet have all the details, the volatility seemed to be a glitch on their end.

I don't have all the details yet, but it seems like this was a glitch on our side and has been fixed in the meantime.

If someone could fix the other 2020-issues, that would be great.

-- 🍌 John 🍌 (@JohnMu) August 11, 2020

The next day, Google's official Webmasters account confirmed that it was, in fact, an indexing glitch and said they had fixed the problem.

On Monday we detected an issue with our indexing systems that affected Google search results. Once the issue was identified, it was promptly fixed by our Site Reliability Engineers and by now it has been mitigated.
Thank you for your patience!

-- Google Webmasters (@googlewmc) August 11, 2020

If you've taken a peek at your ranking and organic search traffic reports and noticed a dip on August 10th that's prompted you to ask "Was there an algorithm update?" the answer is no. The volatility you may have experienced on August 10th was the result of a temporary indexing bug that Google has since fixed.

No action needed on your part.

How Google's indexing system, Caffeine, works

The indexing bug was a good opportunity to explain how Google's indexing system, Caffeine, works.

Google's Gary Illyes took to Twitter to break it down for us.

The indexing system, Caffeine, does multiple things:
1. ingests fetchlogs,
2. renders and converts fetched data,
3. extracts links, meta and structured data,
4. extracts and computes some signals,
5. schedules new crawls,
6. and builds the index that is pushed to serving.

-- Gary 鯨理／경리 Illyes (@methode) August 11, 2020

He explained that the process we know as indexing is actually comprised of multiple steps, such as:

• Ingesting fetchlogs
• Renders and converts fetched data
• Extracts links, meta data, and structured data
• Extracts and computes some signals
• Schedules new crawls
• Builds the index that is pushed to serving

Knowing how much goes into the indexing process makes it easier to understand why something could go wrong on occasion.

If anything goes wrong during the indexing process, there would be downstream effects on rankings, traffic from Google/Organic, and ultimately, your conversions.

That's why it's so important to understand the SEO funnel. Anything that goes wrong at one step has an impact on all following steps.

That's why we can't assume that ranking fluctuations are always the result of an algorithm update or an on-page change we made.

Ranking fluctuations could be the result of an indexing bug, like we saw on August 10th, or because a page was accidentally noindexed or blocked from crawlers.

The difference between indexing bugs and algorithm updates

If you notice volatility in your rankings, it's normal to wonder if there was an algorithm update. However, Google confirms all their Core Algorithm Updates, as they did with their last one: the May 2020 Core Update.

Later today, we are releasing a broad core algorithm update, as we do several times per year. It is called the May 2020 Core Update. Our guidance about such updates remains as we've covered before. Please see this blog post for more about that:https://t.co/e5ZQUAlt0G

-- Google SearchLiaison (@searchliaison) May 4, 2020

So, what's the difference between updates and bugs?

• When Google says that they've released a Core Algorithm Update, that means they have made improvements to their algorithms, which are the systems they use to rank (order) the search results by relevance to the query. If you suffered ranking losses after a Core Update, there may or may not be action you need to take. See "What webmasters should know about Google's core updates" for more information.
• When Google confirms that there was an indexing bug, there's no action you need to take. Indexing issues are the result of an accident on Google's end that they need to fix, rather than a reflection of something you did wrong.

Other reasons your rankings could be fluctuating

Indexing bugs and algorithm updates aren't the only reason your rankings for a query or a group of queries could be fluctuating.

Here are some other reasons why your rankings may have changed:

• Noindex: Accidentally noindexing a page could remove it from Google's index completely. This would cause a drastic change in your rankings, from wherever you were ranking before to 100+ or N/A (depending on what rank tracker you use).
• Blocked: Accidentally blocking a page from being crawled, either in your robots.txt file or by using Google's URL parameters tool, could result in that page disappearing from search.
• On-Site Changes: Your rankings could fluctuate after changing something on your site. For example, removing an internal link to a page from your navigation could cause that page to rank worse, since it doesn't have as much PageRank flowing to it. Technical changes, like going from an HTML framework to a JavaScript framework (content loaded by JS), could cause issues. Additionally, rewriting some or all of the page's content could definitely cause it to rank differently for certain queries.
• Competitors: Sometimes ranking fluctuations aren't the result of something you or Google did. Sometimes, you can lose your position because a competitor wrote a more relevant page for that query.

If your rankings dropped, we recommend looking into all possible options. Ask yourself questions such as:

• Did Google announce a core algorithm update? Check Google Search Liaison or Google Webmasters on Twitter, or the Google Webmaster Central Blog.
• Did Google announce an indexing bug? Check Google Search Liaison or Google Webmasters on Twitter, or the Google Webmaster Central Blog.
• Did I accidentally change any of my directives? Check things like your robots.txt file, noindex tags, status codes and the like to ensure that Google can still crawl your pages, and you're not inadvertently telling them not to index your content.
• Were there changes to the content? Check if your content was rewritten, put in a different format, or otherwise modified.
• Did my competitors publish or update their content? Conduct a Google search for the query you lost position for and see who is ranking well.

You may also be interested in this roundtable discussion with the SEO managers at T-Mobile and Walmart: Algo Updates, Volatility, & How to Roll with the Punches in SEO -- check it out on demand!