Log In Sign Up

unglobalcompact.org
robots.txt

Robots Exclusion Standard data for unglobalcompact.org

Resource Scan

Scan Details

Site Domain unglobalcompact.org
Base Domain unglobalcompact.org
Scan Status Ok
Last Scan2024-09-24T17:16:10+00:00
Next Scan 2024-10-08T17:16:10+00:00

Last Scan

Scanned2024-09-24T17:16:10+00:00
URL https://unglobalcompact.org/robots.txt
Domain IPs 108.157.150.121, 108.157.150.42, 108.157.150.50, 108.157.150.95
Response IP 108.157.150.121
Found Yes
Hash a06d72ec813b331471d4075bf68a56bf60b616250ee684ddcd9461276b9321d5
SimHash 22847e816470

Groups

*

Rule Path
Disallow /COP/analyzing_progress/
Disallow /Cops/
Disallow /docs/
Disallow /news/
Disallow /participant/
Disallow /participants/
Disallow /participants/search
Disallow /search

Other Records

Field Value
crawl-delay 3

seokicks-robot

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
Back to top