bigbookname.com
robots.txt

Robots Exclusion Standard data for bigbookname.com

Resource Scan

Scan Details

Site Domain bigbookname.com
Base Domain bigbookname.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-19T16:17:13+00:00
Next Scan 2024-12-18T16:17:13+00:00

Last Successful Scan

Scanned2024-05-23T06:33:43+00:00
URL https://bigbookname.com/robots.txt
Domain IPs 104.21.82.14, 172.67.150.169
Response IP 172.67.150.169
Found Yes
Hash e5283a8f7dae472d19b29be177f10787d5556f6945c4edcf8874ed9f0cb2cdc4
SimHash c74c54e0af20

Groups

*

Rule Path
Disallow /modal_claim.php
Disallow /modal_claim_post.php
Disallow /modal_info.php
Disallow /modal_show_locked_four.php
Disallow /*access/

googlebot

Rule Path
Disallow /modal_claim.php
Disallow /modal_claim_post.php
Disallow /modal_info.php
Disallow /modal_show_locked_four.php
Disallow /*access/

yandex

Rule Path
Disallow /modal_claim.php
Disallow /modal_claim_post.php
Disallow /modal_info.php
Disallow /modal_show_locked_four.php
Disallow /*access/

applebot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

linguee

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

megaindex.ru
megaindex
megaindex.com

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.