Log In Sign Up

causes.com
robots.txt

Robots Exclusion Standard data for causes.com

Resource Scan

Scan Details

Site Domain causes.com
Base Domain causes.com
Scan Status Ok
Last Scan2024-11-09T20:12:10+00:00
Next Scan 2024-11-16T20:12:10+00:00

Last Scan

Scanned2024-11-09T20:12:10+00:00
URL https://causes.com/robots.txt
Redirect https://www.causes.com/robots.txt
Redirect Domain www.causes.com
Redirect Base causes.com
Domain IPs 108.156.133.105, 108.156.133.36, 108.156.133.58, 108.156.133.74, 2600:9000:2755:3400:15:cdf7:fe00:93a1, 2600:9000:2755:4400:15:cdf7:fe00:93a1, 2600:9000:2755:4c00:15:cdf7:fe00:93a1, 2600:9000:2755:600:15:cdf7:fe00:93a1, 2600:9000:2755:7400:15:cdf7:fe00:93a1, 2600:9000:2755:7e00:15:cdf7:fe00:93a1, 2600:9000:2755:c200:15:cdf7:fe00:93a1, 2600:9000:2755:ee00:15:cdf7:fe00:93a1
Redirect IPs 2600:9000:2755:3000:15:cdf7:fe00:93a1, 2600:9000:2755:4e00:15:cdf7:fe00:93a1, 2600:9000:2755:6400:15:cdf7:fe00:93a1, 2600:9000:2755:7e00:15:cdf7:fe00:93a1, 2600:9000:2755:9a00:15:cdf7:fe00:93a1, 2600:9000:2755:b000:15:cdf7:fe00:93a1, 2600:9000:2755:b200:15:cdf7:fe00:93a1, 2600:9000:2755:c200:15:cdf7:fe00:93a1, 54.230.71.114, 54.230.71.116, 54.230.71.5, 54.230.71.78
Response IP 108.156.133.36
Found Yes
Hash 5c8d4e0773037659992d954358da513dbdc096e64ce959b1d6ab69965488b78a
SimHash a28d2d8d2550

Groups

*

Rule Path
Disallow /users/*
Disallow /view
Back to top

Other Records

Field Value
sitemap https://assets.hosted-assets.com/nextgencauses/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top