creativitydog.it
robots.txt

Robots Exclusion Standard data for creativitydog.it

Resource Scan

Scan Details

Site Domain creativitydog.it
Base Domain creativitydog.it
Scan Status Ok
Last Scan2024-09-30T00:12:59+00:00
Next Scan 2024-10-30T00:12:59+00:00

Last Scan

Scanned2024-09-30T00:12:59+00:00
URL http://creativitydog.it/robots.txt
Domain IPs 37.187.158.205
Response IP 37.187.158.205
Found Yes
Hash 44000586010815827a5f37d89eec228f949c6d49a5e9332028bf2a868ff86f67
SimHash e25d5c40f7f0

Groups

megaindex.ru/2.0
megaindex.ru
megaindex.ru
megaindex.com
megaindex.com/crawler

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 25

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 25

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 25

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • 2 invalid lines.