Log In Sign Up

causes.com
robots.txt

Robots Exclusion Standard data for causes.com

Resource Scan

Scan Details

Site Domain causes.com
Base Domain causes.com
Scan Status Ok
Last Scan2024-05-10T16:59:05+00:00
Next Scan 2024-05-17T16:59:05+00:00

Last Scan

Scanned2024-05-10T16:59:05+00:00
URL https://causes.com/robots.txt
Redirect https://www.causes.com/robots.txt
Redirect Domain www.causes.com
Redirect Base causes.com
Domain IPs 108.156.133.105, 108.156.133.36, 108.156.133.58, 108.156.133.74, 2600:9000:2755:3800:15:cdf7:fe00:93a1, 2600:9000:2755:5000:15:cdf7:fe00:93a1, 2600:9000:2755:5e00:15:cdf7:fe00:93a1, 2600:9000:2755:7000:15:cdf7:fe00:93a1, 2600:9000:2755:a200:15:cdf7:fe00:93a1, 2600:9000:2755:b200:15:cdf7:fe00:93a1, 2600:9000:2755:d600:15:cdf7:fe00:93a1, 2600:9000:2755:e00:15:cdf7:fe00:93a1
Redirect IPs 13.225.103.109, 13.225.103.123, 13.225.103.51, 13.225.103.81, 2600:9000:2755:3c00:15:cdf7:fe00:93a1, 2600:9000:2755:4c00:15:cdf7:fe00:93a1, 2600:9000:2755:6c00:15:cdf7:fe00:93a1, 2600:9000:2755:9c00:15:cdf7:fe00:93a1, 2600:9000:2755:bc00:15:cdf7:fe00:93a1, 2600:9000:2755:c00:15:cdf7:fe00:93a1, 2600:9000:2755:d200:15:cdf7:fe00:93a1, 2600:9000:2755:d600:15:cdf7:fe00:93a1
Response IP 108.156.133.74
Found Yes
Hash 5c8d4e0773037659992d954358da513dbdc096e64ce959b1d6ab69965488b78a
SimHash a28d2d8d2550

Groups

*

Rule Path
Disallow /users/*
Disallow /view
Back to top

Other Records

Field Value
sitemap https://assets.hosted-assets.com/nextgencauses/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top