whois.com
robots.txt

Robots Exclusion Standard data for whois.com

Resource Scan

Scan Details

Site Domain whois.com
Base Domain whois.com
Scan Status Ok
Last Scan2024-09-29T22:09:14+00:00
Next Scan 2024-10-29T22:09:14+00:00

Last Scan

Scanned2024-09-29T22:09:14+00:00
URL https://whois.com/robots.txt
Redirect https://www.whois.com/robots.txt
Redirect Domain www.whois.com
Redirect Base whois.com
Domain IPs 173.199.130.30
Redirect IPs 173.199.130.30
Response IP 173.199.130.30
Found Yes
Hash 897d6dc40256e8517efa3084131420c23feee00d57e89a6faf4f9325ba4667a0
SimHash cb1c8b086b92

Groups

*

Rule Path
Disallow /refresh.php
Disallow /whois/captcha.php
Disallow /whois/refresh.php
Disallow /whois/suggest.php
Disallow /domain-registration/availability.php
Disallow /profile/

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /