dippy.org
robots.txt

Robots Exclusion Standard data for dippy.org

Resource Scan

Scan Details

Site Domain dippy.org
Base Domain dippy.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-23T05:54:25+00:00
Next Scan 2024-10-07T05:54:25+00:00

Last Successful Scan

Scanned2024-08-16T05:53:37+00:00
URL https://dippy.org/robots.txt
Domain IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 151.101.129.91
Found Yes
Hash cb7e5ba0746e6de78e827deb5ed1cac0e51fc3ca6626746a8c59fd26cd26d57d
SimHash 66407097c0b1

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

msn

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

yandex

Rule Path
Disallow /forum2

Other Records

Field Value
crawl-delay 30

naverbot

No rules defined. All paths allowed.

yeti

No rules defined. All paths allowed.

sogou spider

No rules defined. All paths allowed.

youdaobot

No rules defined. All paths allowed.

ichiro

No rules defined. All paths allowed.

baiduspider

Rule Path
Disallow /forum2

Other Records

Field Value
crawl-delay 20

*

No rules defined. All paths allowed.

Warnings

  • 10 invalid lines.