veganstory.com
robots.txt

Robots Exclusion Standard data for veganstory.com

Resource Scan

Scan Details

Site Domain veganstory.com
Base Domain veganstory.com
Scan Status Ok
Last Scan2024-09-15T18:43:44+00:00
Next Scan 2024-09-22T18:43:44+00:00

Last Scan

Scanned2024-09-15T18:43:44+00:00
URL https://veganstory.com/robots.txt
Domain IPs 211.249.222.34
Response IP 211.249.222.34
Found Yes
Hash 0c3d44aebb32d151b7c9744b289b0c65bd32ccbd089948f2818c6ad30030fac2
SimHash 6581d8644ba0

Groups

*

Rule Path
Disallow /guestbook
Disallow /m/guestbook
Disallow /manage
Disallow /owner
Disallow /admin
Disallow /search
Disallow /m/search

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20