rexfeatures.com
robots.txt

Robots Exclusion Standard data for rexfeatures.com

Resource Scan

Scan Details

Site Domain rexfeatures.com
Base Domain rexfeatures.com
Scan Status Ok
Last Scan2024-09-08T09:13:09+00:00
Next Scan 2024-10-08T09:13:09+00:00

Last Scan

Scanned2024-09-08T09:13:09+00:00
URL https://rexfeatures.com/robots.txt
Redirect https://www.rexfeatures.com/robots.txt
Redirect Domain www.rexfeatures.com
Redirect Base rexfeatures.com
Domain IPs 13.33.28.107, 13.33.28.124, 13.33.28.23, 13.33.28.78
Redirect IPs 13.224.163.105, 13.224.163.118, 13.224.163.64, 13.224.163.69
Response IP 13.33.28.124
Found Yes
Hash 01e925057712fa1f91a725dac6b9094a6b660cb1b719171c017019865a784102
SimHash 2844d9154173

Groups

googlebot-image

Rule Path
Disallow /

psbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*.jpg
Disallow /download

bingbot

Rule Path
Disallow /*.jpg
Disallow /download

slurp

Rule Path
Disallow /*.jpg
Disallow /download
Disallow /live
Disallow /livefeed
Disallow /set
Disallow /stacklink
Disallow /login.html
Disallow /staff.html

msnbot

Rule Path
Disallow /*.jpg
Disallow /download

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /download
Disallow /live
Disallow /livefeed
Disallow /set
Disallow /stacklink
Disallow /login.html