aaron-gustafson.com
robots.txt

Robots Exclusion Standard data for aaron-gustafson.com

Resource Scan

Scan Details

Site Domain aaron-gustafson.com
Base Domain aaron-gustafson.com
Scan Status Ok
Last Scan2024-09-08T18:17:01+00:00
Next Scan 2024-10-08T18:17:01+00:00

Last Scan

Scanned2024-09-08T18:17:01+00:00
URL https://aaron-gustafson.com/robots.txt
Redirect https://www.aaron-gustafson.com/robots.txt
Redirect Domain www.aaron-gustafson.com
Redirect Base aaron-gustafson.com
Domain IPs 13.215.144.61, 13.228.199.255
Redirect IPs 13.228.199.255, 52.74.166.77
Response IP 18.139.194.139
Found Yes
Hash 2a6734bdf54bf4c60b3a73efdd88df00cded6d6faba83f85c055f2037a9ddf5b
SimHash 643e5861a583

Groups

*

Rule Path
Disallow /i/c/

applebot
bingbot
baiduspider
duckduckbot
googlebot
slurp
yandexbot

Rule Path
Disallow /clients/

amazonbot
anthropic-ai
ccbot
claude-web
facebookbot
google-extended
gptbot

Rule Path
Disallow /i/
Disallow /m/

Other Records

Field Value
sitemap https://www.aaron-gustafson.com/sitemap.xml