Log In Sign Up

dressafford.com
robots.txt

Robots Exclusion Standard data for dressafford.com

Resource Scan

Scan Details

Site Domain dressafford.com
Base Domain dressafford.com
Scan Status Ok
Last Scan2024-09-10T19:10:07+00:00
Next Scan 2024-10-10T19:10:07+00:00

Last Scan

Scanned2024-09-10T19:10:07+00:00
URL https://dressafford.com/robots.txt
Redirect https://www.dressafford.com/robots.txt
Redirect Domain www.dressafford.com
Redirect Base dressafford.com
Domain IPs 104.21.37.114, 172.67.207.211, 2606:4700:3031::6815:2572, 2606:4700:3035::ac43:cfd3
Redirect IPs 13.33.88.118, 13.33.88.122, 13.33.88.54, 13.33.88.98, 2600:9000:223b:2e00:11:152e:ec0:93a1, 2600:9000:223b:9600:11:152e:ec0:93a1, 2600:9000:223b:a200:11:152e:ec0:93a1, 2600:9000:223b:b800:11:152e:ec0:93a1, 2600:9000:223b:ba00:11:152e:ec0:93a1, 2600:9000:223b:c800:11:152e:ec0:93a1, 2600:9000:223b:cc00:11:152e:ec0:93a1, 2600:9000:223b:de00:11:152e:ec0:93a1
Response IP 13.33.88.122
Found Yes
Hash 639e78ee9e176ea6bd78d22f2f7e190f4013542fda03623b4b7e40393bfd4a7b
SimHash 7114cc72c711

Groups

smtbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow this directory

mj12bot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

*

Rule Path
Allow /
Disallow /articletype/
Disallow /comment/list/*
Allow /comment/list/$
Back to top