ENSURE ALL WEBSITE ASSETS ARE FULLY
CRAWLABLE & INDEXABLE
You must be thinking – why shouldn’t all assets be fully crawlable
and indexable?
Google’s guidelines on this topic state:
“To help Google fully understand your site’s contents, allow all site
assets that would significantly affect page rendering to be crawled:
for example, CSS and JavaScript files that affect the understanding
of the pages. The Google indexing system renders a web page as
the user would see it, including images, CSS, and JavaScript files.
To see which page assets that Googlebot cannot crawl, use the
URL Inspection tool, to debug directives in your robots.txt file, use
the robots.txt Tester tool.”
Well, there are some situations where blocking CSS
(Cascading Stylesheets) and JS (JavaScript) files are
acceptable.First, if you were blocking them because they had issues
playing nice with each other on the server.Second, if you were blocking them because of some
other conflict, either way, Google does have guidelines for
this also.GOOGLE WEBMASTER GUIDELINES: EVERYTHING YOU NEED TO
KNOW & UNDERSTAND