Log In Sign Up

cl.biuky.com
robots.txt

Robots Exclusion Standard data for cl.biuky.com

Resource Scan

Scan Details

Site Domain cl.biuky.com
Base Domain biuky.com
Scan Status Ok
Last Scan2024-09-29T10:04:04+00:00
Next Scan 2024-10-29T10:04:04+00:00

Last Scan

Scanned2024-09-29T10:04:04+00:00
URL https://cl.biuky.com/robots.txt
Domain IPs 104.26.10.80, 104.26.11.80, 172.67.73.169, 2606:4700:20::681a:a50, 2606:4700:20::681a:b50, 2606:4700:20::ac43:49a9
Response IP 104.26.11.80
Found Yes
Hash 45c6c69d27da5d444bf13482705566ac00a67802c7443b6b55439dd4d0e37158
SimHash 0877c1435a90

Groups

baiduspider
ahrefsbot
compraraquibot
voilabot
yandexbot
mj12bot
ezooms
heritrix
bingbot
blexbot
jobdiggerspider
kulturarw3
xovibot
siteexplorer
vagabondo
eccolabot
seokicks
dotbot
ltx71
scrapy
worldwebheritage
semrushbot
semrushbot-sa
prooxibot
applebot
spbot
nekstbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /admin/
Disallow /application/
Disallow /css/
Disallow /data/
Disallow /feeds/
Disallow /imgs/
Disallow /js/
Disallow /lib/
Disallow /plugins/
Disallow /scripts/
Disallow /templates/
Disallow /rtp.php
Allow /

Other Records

Field Value
crawl-delay 30

googlebot
googlebot-news
googlebot-image
googlebot-video
googlebot-mobile
mediapartners
mediapartners-google
adsbot-google

Rule Path
Disallow /admin/
Disallow /application/
Disallow /data/
Disallow /feeds/
Disallow /lib/
Disallow /plugins/
Disallow /scripts/
Disallow /templates/
Disallow /rtp.php
Allow /

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /admin/
Disallow /application/
Disallow /css/
Disallow /data/
Disallow /feeds/
Disallow /imgs/
Disallow /js/
Disallow /lib/
Disallow /plugins/
Disallow /scripts/
Disallow /templates/
Disallow /search/
Disallow /rtp.php
Allow /

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap https://cl.biuky.com/xmls/sitemap.xml
Back to top

Comments

  • Google ignores this, but in case...
Back to top

Warnings

  • 1 invalid line.
Back to top