Log In Sign Up

calorizator.ru
robots.txt

Robots Exclusion Standard data for calorizator.ru

Resource Scan

Scan Details

Site Domain calorizator.ru
Base Domain calorizator.ru
Scan Status Ok
Last Scan2024-06-09T14:57:18+00:00
Next Scan 2024-06-16T14:57:18+00:00

Last Scan

Scanned2024-06-09T14:57:18+00:00
URL https://calorizator.ru/robots.txt
Domain IPs 104.21.31.193, 172.67.179.155, 2606:4700:3033::6815:1fc1, 2606:4700:3034::ac43:b39b
Response IP 104.21.31.193
Found Yes
Hash b89f31d3a1b83a743c3c2d77ea1c5de01686a148341420be8cbfb388cbbd8d3e
SimHash bc949509cd74

Groups

yandex

Rule Path
Disallow /includes/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /sites/all/
Disallow /themes/
Disallow /s-x-d/
Disallow /line/
Disallow /widgets/
Disallow /*.txt
Disallow /cron.php
Disallow /install.php
Disallow /update.php
Disallow /xmlrpc.php
Disallow /admin
Disallow /admin/
Disallow /comment/reply
Disallow /comment/reply/
Disallow /contact
Disallow /contact/
Disallow /logout
Disallow /logout/
Disallow /node
Disallow /node/
Disallow /search
Disallow /search/
Disallow /user/*
Disallow /taxonomy/term
Disallow /taxonomy/term/
Disallow /analyzer/body1
Disallow /analyzer/body1/
Disallow /analyzer/body2
Disallow /analyzer/body2/
Disallow /product/choice
Disallow /product/choice/
Disallow /product/*order%3D*%26sort%3D*
Disallow /product/*form_build_id%3D*%26form_id%3D*
Disallow /recipe?order=*&sort=*
Disallow /recipes/*/*order%3D*%26sort%3D*
Disallow /recipe2
Disallow /recipe2/
Disallow /recept
Disallow /recept/
Allow /*.js*
Allow /*.css*
Allow /*.jpg
Allow /*.gif
Allow /*.png

*

Rule Path
Disallow /includes/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /sites/all/
Disallow /themes/
Disallow /s-x-d/
Disallow /line/
Disallow /widgets/
Disallow *.txt
Disallow /cron.php
Disallow /install.php
Disallow /update.php
Disallow /xmlrpc.php
Disallow /admin
Disallow /admin/
Disallow /comment/reply
Disallow /comment/reply/
Disallow /contact
Disallow /contact/
Disallow /logout
Disallow /logout/
Disallow /node
Disallow /node/
Disallow /search
Disallow /search/
Disallow /user/register
Disallow /user/register/
Disallow /user/password
Disallow /user/password/
Disallow /user/login
Disallow /user/login/
Disallow /taxonomy/term
Disallow /taxonomy/term/
Disallow /?q=*
Disallow /analyzer/body1
Disallow /analyzer/body1/
Disallow /analyzer/body2
Disallow /analyzer/body2/
Disallow /product/choice
Disallow /product/choice/
Disallow /product/*order%3D*%26sort%3D*
Disallow /product/*form_build_id%3D*%26form_id%3D*
Disallow /recipe?order=*&sort=*
Disallow /recipes/*/*order%3D*%26sort%3D*
Disallow /recipe2
Disallow /recipe2/
Disallow /recept
Disallow /recept/
Allow /*.js*
Allow /*.css*
Allow /*.jpg
Allow /*.gif
Allow /*.png

yadirectbot

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow
Back to top

Comments

  • $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $
  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Disallow: /misc/
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top