Log In Sign Up

islaguru.com
robots.txt

Robots Exclusion Standard data for islaguru.com

Resource Scan

Scan Details

Site Domain islaguru.com
Base Domain islaguru.com
Scan Status Ok
Last Scan2025-10-17T11:52:24+00:00
Next Scan 2025-10-24T11:52:24+00:00

Last Scan

Scanned2025-10-17T11:52:24+00:00
URL https://islaguru.com/robots.txt
Redirect https://www.islaguru.com/robots.txt
Redirect Domain www.islaguru.com
Redirect Base islaguru.com
Domain IPs 75.2.70.75, 99.83.190.102
Redirect IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 34.202.203.47
Found Yes
Hash cfd4ab33c236ae03980f637d2487799bac4e8c714b89ee5a100b416744bafce1
SimHash 7de05af7ecb4

Groups

*

Rule Path
Disallow /admin/
Disallow /login/
Disallow /register/
Disallow /private/
Disallow /search
Disallow /*?edit
Allow /images/
Allow /css/
Allow /js/
Allow /assets/
Allow /category/
Allow /tag/
Back to top

Other Records

Field Value
sitemap https://islaguru.com/sitemap.xml
sitemap https://www.islaguru.com/sitemap.xml
Back to top

Comments

  • Block internal admin or backend directories (harmless on Webflow but fine to keep)
  • Webflow system/utility URLs
  • Keep important resources crawlable for rendering
  • Explicitly allow taxonomy pages (not strictly needed, but kept per your preference)
  • Sitemap
Back to top