Log In Sign Up

mangarw.com
robots.txt

Robots Exclusion Standard data for mangarw.com

Resource Scan

Scan Details

Site Domain mangarw.com
Base Domain mangarw.com
Scan Status Ok
Last Scan2025-11-28T15:38:21+00:00
Next Scan 2025-12-05T15:38:21+00:00

Last Scan

Scanned2025-11-28T15:38:21+00:00
URL https://mangarw.com/robots.txt
Domain IPs 104.21.88.124, 172.67.179.93, 2606:4700:3034::6815:587c, 2606:4700:3034::ac43:b35d
Response IP 104.21.88.124
Found Yes
Hash 1e9ae60f4661084f6183f41cc0fd12c04c0e05577b9bb58c21787a61e714e078
SimHash a0065990e500

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /api/admin/
Disallow /user/
Disallow /profile/
Disallow /search?
Disallow /browse?*q=
Allow /css/
Allow /js/
Allow /img/
Disallow /test/
Disallow /temp/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://mangarw.com/sitemap.xml
Back to top

Comments

  • Robots.txt for MangaRaw
  • Sitemaps
  • Crawl delay
  • Block admin pages
  • Block user-specific pages
  • Block search results
  • Allow search engines to access CSS and JS
  • Block temporary or test pages
Back to top