Log In Sign Up

zueriost.ch
robots.txt

Robots Exclusion Standard data for zueriost.ch

Resource Scan

Scan Details

Site Domain zueriost.ch
Base Domain zueriost.ch
Scan Status Ok
Last Scan2024-11-10T04:02:55+00:00
Next Scan 2024-11-17T04:02:55+00:00

Last Scan

Scanned2024-11-10T04:02:55+00:00
URL https://zueriost.ch/robots.txt
Domain IPs 157.230.27.144
Response IP 157.230.27.144
Found Yes
Hash aab851e8395adfcf9f796c11fc4e1e037ec7d8c5c80b4431a575d0278774aa6b
SimHash b8169d4a4754

Groups

*

Rule Path
Allow /
Disallow /nachrichten
Disallow /suche?*
Disallow /*?s=*

psbot
yandex
petalbot
mail.ru_bot
megaindex
baiduspider
yisouspider
bytespider
sogou web spider
sogou inst spider
proximic
admantx
seekport crawler
semrushbot
blexbot
mj12bot
dotbot
gptbot
ccbot
google-extended

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://zueriost.ch/sitemap.xml
sitemap https://auth.zueriost.ch/api/xml/news-sitemap
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • Disallow predefined bots/crawlers
Back to top

Warnings

  • 1 invalid line.
Back to top