Log In Sign Up

autosearch.sk
robots.txt

Robots Exclusion Standard data for autosearch.sk

Resource Scan

Scan Details

Site Domain autosearch.sk
Base Domain autosearch.sk
Scan Status Ok
Last Scan2024-11-04T21:14:33+00:00
Next Scan 2024-12-04T21:14:33+00:00

Last Scan

Scanned2024-11-04T21:14:33+00:00
URL https://autosearch.sk/robots.txt
Redirect https://www.autosearch.sk/robots.txt
Redirect Domain www.autosearch.sk
Redirect Base autosearch.sk
Domain IPs 2a01:4f8:201:214d::2, 78.46.18.238
Redirect IPs 2a01:4f8:201:214d::2, 78.46.18.238
Response IP 78.46.18.238
Found Yes
Hash 0e817341d896acf6f31013401b6537d9d98c8fec8cbcddd9c297058c15cc64aa
SimHash 501a94e0e170

Groups

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

discobot

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

babya discoverer

Rule Path
Disallow /

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*

Other Records

Field Value
crawl-delay 5

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /cache/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /tmp/
Disallow /xmlrpc/
Disallow *?format=feed&type=rss
Disallow *?format=feed&type
Allow /components/com_djclassifieds/assets/images/
Allow /components/com_djclassifieds/images/
Allow /components/com_djclassifieds/assets/moocomplete/
Allow /components/com_djclassifieds/images/item/
Allow /components/com_djclassifieds/themes/jm-car-classifieds/css/
Allow /components/com_djmediatools/assets/images/
Allow /components/com_djmediatools/layouts/slideshow/images/
Allow /components/com_djmediatools/layouts/slideshow/js/
Allow /modules/mod_djmegamenu/assets/js/
Allow /plugins/system/ef4_jmframework/includes/assets/template/js/
Allow /templates/jm-car-classifieds/cache/
Allow /templates/jm-car-classifieds/js/
Allow /templates/jm-car-classifieds/images/
Allow /components/com_djclassifieds/assets/picbox/css/
Allow /components/com_djclassifieds/assets/picbox/js/
Back to top

Other Records

Field Value
sitemap http://www.autosearch.sk/sitemap.xml
Back to top

Comments

  • Block MJ12bot as it is just noise
  • Block Sogou
  • Block SEOkicks
  • SEOkicks
  • Dicoveryengine.com
  • Blekkobot
  • Block BlexBot
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block Perl LWP
  • Block netEstate NE Crawler
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Exabot
  • Yandex
  • Babya Discoverer
Back to top

Warnings

  • 2 invalid lines.
Back to top