cdga.org
robots.txt

Robots Exclusion Standard data for cdga.org

Resource Scan

Scan Details

Site Domain cdga.org
Base Domain cdga.org
Scan Status Ok
Last Scan2024-05-05T16:47:11+00:00
Next Scan 2024-05-12T16:47:11+00:00

Last Scan

Scanned2024-05-05T16:47:11+00:00
URL https://cdga.org/robots.txt
Redirect https://www.cdga.org/robots.txt
Redirect Domain www.cdga.org
Redirect Base cdga.org
Domain IPs 69.41.141.27
Redirect IPs 69.41.141.27
Response IP 69.41.141.27
Found Yes
Hash 1f56c0d8ffb26aa8964364f847f1da98a0fb708654641eae97fc8af31fecd48f
SimHash a5065872c9d3

Groups

*

Rule Path
Disallow /apis/
Disallow /css/
Disallow /error/
Disallow /incs/
Disallow /js/
Disallow /libs/
Disallow /scss/
Disallow /sitedown/

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

nutch

Rule Path
Disallow /