Google has updated its crawler and fetcher documentation to include a new section on HTTP caching, clarifying how its crawlers handle cache control headers. Alongside this, Gary Illyes from Google published a blog post titled HTTP Caching, urging site owners to enable caching for their web pages.

What’s New?

The updated documentation explains:

“Google’s crawling infrastructure supports heuristic HTTP caching as defined by the HTTP caching standard, specifically through the ETag response- and If-None-Match request header, and the Last-Modified response- and If-Modified-Since request header.”

However, Google’s crawlers do not support other HTTP caching directives. When both ETag and Last-Modifiedresponse headers are present, Google’s crawlers prioritize the ETag value.

A Decline in Caching

Illyes highlighted in his blog post that the proportion of cacheable requests has decreased over the past decade.

• 10 years ago: Only 0.026% of total fetches were cacheable—a modest figure at best.
• Today: That number has dropped even further to 0.017%.

The Benefits of Caching

Illyes encouraged site owners to enable HTTP caching, emphasizing user experience and cost savings:

“If you’re in the business of making your users happy and perhaps also want to potentially save a few bucks on your hosting bill, talk to your hosting or CMS provider, or your developers about how to enable HTTP caching for your site. If nothing else, your users will like you a bit more.”

Why It Matters

While enabling caching can help Google crawl your site more efficiently, there’s no direct indication of SEO, ranking, or crawl budget benefits. Nonetheless, improving crawl efficiency might enhance your site’s performance and ensure happier users—and potentially, a happier Googlebot.

Would you like help optimizing this further?

If you’re still feeling overwhelmed or unsure, let the experts handle it for you! Check out our monthly SEO packages and leave the hard work to us.