Log In Sign Up

wtca.org
robots.txt

Robots Exclusion Standard data for wtca.org

Resource Scan

Scan Details

Site Domain wtca.org
Base Domain wtca.org
Scan Status Ok
Last Scan2024-08-28T06:53:08+00:00
Next Scan 2024-09-27T06:53:08+00:00

Last Scan

Scanned2024-08-28T06:53:08+00:00
URL https://wtca.org/robots.txt
Redirect https://www.wtca.org/robots.txt
Redirect Domain www.wtca.org
Redirect Base wtca.org
Domain IPs 108.157.142.14, 108.157.142.29, 108.157.142.33, 108.157.142.9, 2600:9000:24f8:4400:8:5460:3780:93a1, 2600:9000:24f8:5800:8:5460:3780:93a1, 2600:9000:24f8:6200:8:5460:3780:93a1, 2600:9000:24f8:6400:8:5460:3780:93a1, 2600:9000:24f8:b200:8:5460:3780:93a1, 2600:9000:24f8:b400:8:5460:3780:93a1, 2600:9000:24f8:f200:8:5460:3780:93a1, 2600:9000:24f8:f600:8:5460:3780:93a1
Redirect IPs 13.227.254.19, 13.227.254.46, 13.227.254.52, 13.227.254.83, 2600:9000:200a:1400:5:6bcb:76c0:93a1, 2600:9000:200a:1600:5:6bcb:76c0:93a1, 2600:9000:200a:1e00:5:6bcb:76c0:93a1, 2600:9000:200a:400:5:6bcb:76c0:93a1, 2600:9000:200a:4800:5:6bcb:76c0:93a1, 2600:9000:200a:4c00:5:6bcb:76c0:93a1, 2600:9000:200a:5c00:5:6bcb:76c0:93a1, 2600:9000:200a:9a00:5:6bcb:76c0:93a1
Response IP 13.227.254.52
Found Yes
Hash 6a5bd64a6f0a3d2397606dd937735e99516044bdd7a1da0da0b876cd8ecdaf8a
SimHash b285298d6550

Groups

*

Rule Path
Disallow /jobs/
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top