×
Wednesday, February 28, 2024

Google Says Sometimes Bot Protection Services Serve noindex - Search Engine Roundtable

Last updated Thursday, December 28, 2023 08:02 ET , Source: NewsService

Did you know that sometimes those bot protection services will serve Google and other search engines noindex directives? Google's John Mueller said this on X, saying, "Sometimes there's bot-protection (or a login, interstitial, etc) triggering that has a noindex on it."

He also added that it would be better for them to serve 503 server status codes. "Better would be to use 503 for server-side blocks like bot-protection," he wrote.

In the end, the person he replied to was having a different issue causing the noindex, it was not bot protection. He said it was "some shenanigans with NextJS doing client side rendering instead of server rendering." "The HTML had a noindex tag, which was removed after rendering on the browser. The HTML shown on GSC is the one AFTER JavaScript rendering, which is why I couldn’t find it and thought it was a GSC bug," he added.

This was the error he was seeing:

wtf is going on here? why is Google not indexing this one?

They say there's a "noindex" tag, but there isn't one. robots.txt looks good too.

Anyone? #seo help pic.twitter.com/DYzJsLQ4Yk

— Guilherme (@goenning) December 26, 2023

Thanks John, eventually found it thanks to @HamilcDev

Some shenanigans with NextJS doing client side rendering instead of server rendering

The HTML had a noindex tag, which was removed after rendering on the browser. The HTML shown on GSC is the one AFTER JavaScript rendering,… https://t.co/NffU7suVLM

— Guilherme (@goenning) December 26, 2023

Forum discussion at X.



Read Full Story: https://news.google.com/rss/articles/CBMiVGh0dHBzOi8vd3d3LnNlcm91bmR0YWJsZS5jb20vZ29vZ2xlLWJvdC1wcm90ZWN0aW9uLXNlcnZpY2VzLXNlcnZlLW5vaW5kZXgtMzY2MjQuaHRtbNIBAA?oc=5

Your content is great. However, if any of the content contained herein violates any rights of yours, including those of copyright, please contact us immediately by e-mail at media[@]kissrpr.com.