grab.by
robots.txt

Robots Exclusion Standard data for grab.by

Resource Scan

Scan Details

Site Domain grab.by
Base Domain grab.by
Scan Status Ok
Last Scan2024-08-29T19:39:21+00:00
Next Scan 2024-09-28T19:39:21+00:00

Last Scan

Scanned2024-08-29T19:39:21+00:00
URL https://grab.by/robots.txt
Redirect https://www.grab.by/robots.txt
Redirect Domain www.grab.by
Redirect Base grab.by
Domain IPs 104.21.66.117, 172.67.159.152, 2606:4700:3032::ac43:9f98, 2606:4700:3036::6815:4275
Redirect IPs 104.21.66.117, 172.67.159.152, 2606:4700:3032::ac43:9f98, 2606:4700:3036::6815:4275
Response IP 104.21.66.117
Found Yes
Hash 7a65d6fbff3a3fdeae19e21d31cf0a14278e723a358cd9ec6f6292898b48e751
SimHash e9209a008bb2

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.grab.by/sitemap.xml
sitemap https://www.grab.by/sitemap.rss