annieskitclubs.com
robots.txt

Robots Exclusion Standard data for annieskitclubs.com

Resource Scan

Scan Details

Site Domain annieskitclubs.com
Base Domain annieskitclubs.com
Scan Status Ok
Last Scan2024-09-21T23:49:04+00:00
Next Scan 2024-09-28T23:49:04+00:00

Last Scan

Scanned2024-09-21T23:49:04+00:00
URL https://annieskitclubs.com/robots.txt
Redirect https://www.annieskitclubs.com/robots.txt
Redirect Domain www.annieskitclubs.com
Redirect Base annieskitclubs.com
Domain IPs 208.40.241.72
Redirect IPs 208.40.241.72
Response IP 208.40.241.72
Found Yes
Hash ce76da95a25b954d76aa1baeb0613277c6c41afcd3ae667e6736d143b7a1ef75
SimHash 08544a90a2c1

Groups

baiduspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

dealozbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /account/

bingbot

Rule Path
Disallow /account/

bingshoppingbot

Rule Path
Disallow /account/

*

Rule Path
Disallow

news.grets.store

Rule Path
Disallow /

static.seders.website

Rule Path
Disallow /

info.seders.website

Rule Path
Disallow /

rida.tokyo

Rule Path
Disallow /

kar.razas.site

Rule Path
Disallow /

game.fertuk.site

Rule Path
Disallow /

ofer.bartikus.site

Rule Path
Disallow /

trast.mantero.online

Rule Path
Disallow /

garold.dertus.site

Rule Path
Disallow /

games.patlik.site

Rule Path
Disallow /

geedoproductsearch

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10