anuko.com
robots.txt

Robots Exclusion Standard data for anuko.com

Resource Scan

Scan Details

Site Domain anuko.com
Base Domain anuko.com
Scan Status Ok
Last Scan2024-06-20T03:20:56+00:00
Next Scan 2024-07-20T03:20:56+00:00

Last Scan

Scanned2024-06-20T03:20:56+00:00
URL https://anuko.com/robots.txt
Redirect https://www.anuko.com:443/robots.txt
Redirect Domain www.anuko.com
Redirect Base anuko.com
Domain IPs 167.99.179.49
Redirect IPs 167.99.179.49
Response IP 167.99.179.49
Found Yes
Hash 6255194e6c743b49bf335317c5b0c8bcd5a7ef2df2f4524026e8700004512bf5
SimHash dedc8ad08681

Groups

*

Rule Path
Disallow /download/
Disallow /downloads/
Disallow /mobile/

adsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

awariobot
awariorssbot
awariosmartbot

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

daum

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

df bot 1.0

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

freefind

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

gowikibot

Rule Path
Disallow /

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

icc-crawler

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

linguee

Rule Path
Disallow /

macocu

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

orbbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

tombapublicwebcrawler

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /