Log In Sign Up

wtca.org
robots.txt

Robots Exclusion Standard data for wtca.org

Resource Scan

Scan Details

Site Domain wtca.org
Base Domain wtca.org
Scan Status Ok
Last Scan2024-10-27T06:53:24+00:00
Next Scan 2024-11-26T06:53:24+00:00

Last Scan

Scanned2024-10-27T06:53:24+00:00
URL https://wtca.org/robots.txt
Redirect https://www.wtca.org/robots.txt
Redirect Domain www.wtca.org
Redirect Base wtca.org
Domain IPs 108.157.142.14, 108.157.142.29, 108.157.142.33, 108.157.142.9, 2600:9000:26f2:1200:8:5460:3780:93a1, 2600:9000:26f2:2200:8:5460:3780:93a1, 2600:9000:26f2:2c00:8:5460:3780:93a1, 2600:9000:26f2:2e00:8:5460:3780:93a1, 2600:9000:26f2:400:8:5460:3780:93a1, 2600:9000:26f2:5a00:8:5460:3780:93a1, 2600:9000:26f2:9c00:8:5460:3780:93a1, 2600:9000:26f2:d600:8:5460:3780:93a1
Redirect IPs 13.227.254.19, 13.227.254.46, 13.227.254.52, 13.227.254.83, 2600:9000:200a:1200:5:6bcb:76c0:93a1, 2600:9000:200a:600:5:6bcb:76c0:93a1, 2600:9000:200a:7800:5:6bcb:76c0:93a1, 2600:9000:200a:7c00:5:6bcb:76c0:93a1, 2600:9000:200a:8c00:5:6bcb:76c0:93a1, 2600:9000:200a:ce00:5:6bcb:76c0:93a1, 2600:9000:200a:d200:5:6bcb:76c0:93a1, 2600:9000:200a:d400:5:6bcb:76c0:93a1
Response IP 13.227.254.46
Found Yes
Hash 6a5bd64a6f0a3d2397606dd937735e99516044bdd7a1da0da0b876cd8ecdaf8a
SimHash b285298d6550

Groups

*

Rule Path
Disallow /jobs/
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top