Google Shares Insights into Indexing & Crawl Budget - Search Engine Journal

Google shares insights on factors that determine whether content is indexed

Unlock and Access Your Most Valuable SEO Data

Replace (not provided) with ALL of your organic keywords inside of Adobe & Google Analytics. Analyze performance by 400+ dimensions and metrics.

TRY FOR FREE

Google recently published a podcast discussing what’s known as a crawl budget and what influences Google to index content.

Both Gary Illyes and Martin Splitt shared insights into indexing the web, as understood from Google’s perspective.

Origin of the Crawl Budget Concept

Gary Illyes said that the concept of a crawl budget was something created outside of Google by the search community.

He explained there wasn’t any one thing internally within Google that corresponded with the idea of a crawl budget.

When people talked about a crawl budget, what was happening inside Google involved multiple metrics, not this one thing called a crawl budget.

So inside Google they talked about what could represent a crawl budget and came up with a way of talking about it.

He said:

“…for the longest time we were saying that we don’t have the concept of crawl budget. And it was true.

We didn’t have something that could mean crawl budget on its own- the same way we don’t have a number for EAT, for example.

And then, because people were talking about it, we tried to come up with something… at least, somehow defined.

And then we worked with two or three or four teams– I don’t remember– where we tried to come up with...

Your content is great. However, if any of the content contained herein violates any rights of yours, including those of copyright, please contact us immediately by e-mail at media[@]kissrpr.com.