cabdirect.org
robots.txt

Robots Exclusion Standard data for cabdirect.org

Resource Scan

Scan Details

Site Domain cabdirect.org
Base Domain cabdirect.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-14T10:43:34+00:00
Next Scan 2024-06-13T10:43:34+00:00

Last Successful Scan

Scanned2024-01-23T08:26:03+00:00
URL https://www.cabdirect.org/robots.txt
Domain IPs 46.37.187.200
Response IP 46.37.187.200
Found Yes
Hash c0de885c16cd3afde8ad59840b9482d50daf4bfc52eb99d6ecea3ac016cd53e4
SimHash c81441528133

Groups

*

Rule Path
Allow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

redbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cabdirect.org/cabdirect/injectlist_v2.xml

Warnings

  • 2 invalid lines.