craftylife.net
robots.txt

Robots Exclusion Standard data for craftylife.net

Resource Scan

Scan Details

Site Domain craftylife.net
Base Domain craftylife.net
Scan Status Ok
Last Scan2024-09-22T15:27:39+00:00
Next Scan 2024-10-22T15:27:39+00:00

Last Scan

Scanned2024-09-22T15:27:39+00:00
URL https://craftylife.net/robots.txt
Domain IPs 192.124.249.6
Response IP 192.124.249.6
Found Yes
Hash 84dcdd174d918f2720a67b15b6d4acb9c8d7995dc536226553ee4483f983b69e
SimHash 0d1ed99182b3

Groups

baiduspider

Rule Path
Disallow /

baiduimagespider+(+http://www.baidu.jp/search/s308.html)

Rule Path
Disallow /

baiduspider+(+http://help.baidu.jp/system/05.html)

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider_jp.html)

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

*

Rule Path
Disallow /wp-login.php
Disallow /*.gif$
Disallow /*.Gif$
Disallow /*.GIF$
Disallow /*.jpg$
Disallow /*.Jpg$
Disallow /*.JPG$
Disallow /*.jpeg$
Disallow /*.Jpeg$
Disallow /*.JPEG$
Disallow /*.pdf$
Disallow /*.Pdf$
Disallow /*.PDF$
Disallow /*.zip$
Disallow /*.Zip$
Disallow /*.ZIP$

Other Records

Field Value
crawl-delay 30.0