setmore.com
robots.txt

Robots Exclusion Standard data for setmore.com

Resource Scan

Scan Details

Site Domain setmore.com
Base Domain setmore.com
Scan Status Ok
Last Scan2024-05-16T01:50:35+00:00
Next Scan 2024-06-15T01:50:35+00:00

Last Scan

Scanned2024-05-16T01:50:35+00:00
URL https://setmore.com/robots.txt
Redirect https://www.setmore.com/robots.txt
Redirect Domain www.setmore.com
Redirect Base setmore.com
Domain IPs 216.239.32.21, 216.239.34.21, 216.239.36.21, 216.239.38.21
Redirect IPs 34.95.113.25
Response IP 34.95.113.25
Found Yes
Hash a71ec7548624fa20a99d9bb84782abd9ae3749b8fb88c458e86bdde8538fbdd1
SimHash 455c5f65c753

Groups

googlebot

Rule Path
Allow /
Disallow /s/
Disallow /lp/
Disallow /jsp/
Disallow /demo/
Disallow /stageblog/
Disallow /sm-no-follow-pages.html

proximic

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.setmore.com/sitemap.xml

Comments

  • Allow Googlebot
  • Block Comscore Web Crawler