netcaucus.org
robots.txt

Robots Exclusion Standard data for netcaucus.org

Resource Scan

Scan Details

Site Domain netcaucus.org
Base Domain netcaucus.org
Scan Status Ok
Last Scan2024-05-25T18:22:27+00:00
Next Scan 2024-06-24T18:22:27+00:00

Last Scan

Scanned2024-05-25T18:22:27+00:00
URL https://netcaucus.org/robots.txt
Domain IPs 104.26.14.81, 104.26.15.81, 172.67.68.206, 2606:4700:20::681a:e51, 2606:4700:20::681a:f51, 2606:4700:20::ac43:44ce
Response IP 104.26.14.81
Found Yes
Hash cdc7508531881dafdafd41319193ca662234ea85cd05530c204c4966fd898880
SimHash 26a0510e86b6

Groups

*

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 3

Comments

  • This file sets out restrictions that most spiders and automatic
  • web-indexers voluntarily abide by. For more information, check out:
  • http://info.webcrawler.com/mak/projects/robots/norobots.html