Log In Sign Up

scifi.sk
robots.txt

Robots Exclusion Standard data for scifi.sk

Resource Scan

Scan Details

Site Domain scifi.sk
Base Domain scifi.sk
Scan Status Ok
Last Scan2024-11-15T19:32:22+00:00
Next Scan 2024-11-22T19:32:22+00:00

Last Scan

Scanned2024-11-15T19:32:22+00:00
URL https://scifi.sk/robots.txt
Redirect https://www.scifi.sk/robots.txt
Redirect Domain www.scifi.sk
Redirect Base scifi.sk
Domain IPs 158.197.16.67
Redirect IPs 158.197.16.67, 2001:4118:400:10:5054:ff:fe7c:ebac
Response IP 158.197.16.67
Found Yes
Hash 89c46ef814e977ff7d11f46be0fff42212f1f5ae0fcf8f3646877a97016c6e5c
SimHash a8149d0ac674

Groups

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /podporte/platba/

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Directories
  • Disallow: /adm/
  • Files
  • Disallow: /CHANGELOG.txt
  • URLs
Back to top