defencenews.sk
robots.txt

Robots Exclusion Standard data for defencenews.sk

Resource Scan

Scan Details

Site Domain defencenews.sk
Base Domain defencenews.sk
Scan Status Ok
Last Scan2024-11-10T15:22:06+00:00
Next Scan 2024-11-17T15:22:06+00:00

Last Scan

Scanned2024-11-10T15:22:06+00:00
URL https://defencenews.sk/robots.txt
Redirect https://www.defencenews.sk/robots.txt
Redirect Domain www.defencenews.sk
Redirect Base defencenews.sk
Domain IPs 217.67.31.48
Redirect IPs 217.67.31.48
Response IP 217.67.31.48
Found Yes
Hash 68a7d107249241e6593eaf730f453ef451878c4422544134b2ab7ef7ab1d7efe
SimHash b819cce64a73

Groups

*

Rule Path
Disallow /index.php
Disallow /*.asp?strana=*
Disallow /tlac.asp
Disallow /stats/
Disallow /dennik/*.html
Disallow /spravy/*.html
Disallow /foto.asp?galerie=*
Disallow /aarticle.asp
Disallow /typo3temp/
Disallow /ajax/
Allow /typo3temp/sitemap/

sentibot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

grapeshot

Rule Path
Disallow

Comments

  • http://sentibot.eu/
  • http://corpora.informatik.uni-leipzig.de/crawler_faq.html
  • http://www.grapeshot.co.uk/crawler.php