woodstockoutlet.com
robots.txt

Robots Exclusion Standard data for woodstockoutlet.com

Resource Scan

Scan Details

Site Domain woodstockoutlet.com
Base Domain woodstockoutlet.com
Scan Status Ok
Last Scan2024-09-25T16:10:35+00:00
Next Scan 2024-10-25T16:10:35+00:00

Last Scan

Scanned2024-09-25T16:10:35+00:00
URL https://woodstockoutlet.com/robots.txt
Redirect https://www.woodstockoutlet.com/robots.txt
Redirect Domain www.woodstockoutlet.com
Redirect Base woodstockoutlet.com
Domain IPs 54.210.26.57
Redirect IPs 54.210.26.57
Response IP 54.210.26.57
Found Yes
Hash f3f519a3507182d59525da706c3236278fa047302edd2b06b9d63c559fab74ce
SimHash 64559c747383

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /junk/
Disallow /*.aspx$

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /