Log In Sign Up

remine.com
robots.txt

Robots Exclusion Standard data for remine.com

Resource Scan

Scan Details

Site Domain remine.com
Base Domain remine.com
Scan Status Ok
Last Scan2024-05-20T02:07:40+00:00
Next Scan 2024-06-19T02:07:40+00:00

Last Scan

Scanned2024-05-20T02:07:40+00:00
URL https://remine.com/robots.txt
Redirect https://www.remine.com/robots.txt
Redirect Domain www.remine.com
Redirect Base remine.com
Domain IPs 18.164.174.120, 18.164.174.129, 18.164.174.20, 18.164.174.8
Redirect IPs 2600:9000:2475:2400:0:9913:2380:93a1, 2600:9000:2475:2a00:0:9913:2380:93a1, 2600:9000:2475:3000:0:9913:2380:93a1, 2600:9000:2475:5200:0:9913:2380:93a1, 2600:9000:2475:5e00:0:9913:2380:93a1, 2600:9000:2475:7200:0:9913:2380:93a1, 2600:9000:2475:8800:0:9913:2380:93a1, 2600:9000:2475:b400:0:9913:2380:93a1, 65.9.112.107, 65.9.112.114, 65.9.112.30, 65.9.112.71
Response IP 108.157.60.87
Found Yes
Hash 72fa474ff02c49f47d77818dc852263e481591bd21d1cab61021c486e6a201a9
SimHash 69911a708691

Groups

adsbot-google
adsbot-google-mobile
adsbot-google-mobile-android

Rule Path
Allow /*/land/

*

Rule Path
Disallow /healthz

baiduspider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://remine.com/sitemaps/en-us/states_or_provinces_sitemap.xml
Back to top

Comments

  • Whitelist
  • Healthcheck
  • Baidu Search Engine
  • site map
Back to top