Log In Sign Up

episode.ninja
robots.txt

Robots Exclusion Standard data for episode.ninja

Resource Scan

Scan Details

Site Domain episode.ninja
Base Domain episode.ninja
Scan Status Ok
Last Scan2024-09-20T18:08:37+00:00
Next Scan 2024-09-27T18:08:37+00:00

Last Scan

Scanned2024-09-20T18:08:37+00:00
URL https://episode.ninja/robots.txt
Domain IPs 76.76.21.21
Response IP 76.76.21.21
Found Yes
Hash 5279b901c6d40c56a80047d6df77245b5e43cbb6131cc1cb004d007ec43777b8
SimHash 0a05091c06d2

Groups

*

Rule Path
Disallow /*.json$
Disallow /*_buildManifest.js$
Disallow /*_middlewareManifest.js$
Disallow /*_ssgManifest.js$
Disallow /*.js$
Back to top

Other Records

Field Value
sitemap https://episode.ninja/sitemap.xml
Back to top

Comments

  • Next.JS Crawl Budget Performance Updates
  • Block files ending in .json, _buildManifest.js, _middlewareManifest.js, _ssgManifest.js, and any other JS files
  • The asterisks allows any file name
  • The dollar sign ensures it only matches the end of an URL and not a oddly formatted url (e.g. /locations.json.html)
Back to top