Log In Sign Up

objetivocastillalamancha.es
robots.txt

Robots Exclusion Standard data for objetivocastillalamancha.es

Resource Scan

Scan Details

Site Domain objetivocastillalamancha.es
Base Domain objetivocastillalamancha.es
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-12T08:31:27+00:00
Next Scan 2024-12-11T08:31:27+00:00

Last Successful Scan

Scanned2024-05-16T00:54:58+00:00
URL https://objetivocastillalamancha.es/robots.txt
Domain IPs 192.250.229.28
Response IP 192.250.229.28
Found Yes
Hash f20bb7c0b13b7bdc898c0430c50f00b4921f75c7cd9f78e7517389a72c42ac4a
SimHash da045b584d71

Groups

*

Rule Path
Allow /misc/*.css$
Allow /misc/*.css?
Allow /misc/*.js$
Allow /misc/*.js?
Allow /misc/*.gif
Allow /misc/*.jpg
Allow /misc/*.jpeg
Allow /misc/*.png
Allow /modules/*.css$
Allow /modules/*.css?
Allow /modules/*.js$
Allow /modules/*.js?
Allow /modules/*.gif
Allow /modules/*.jpg
Allow /modules/*.jpeg
Allow /modules/*.png
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add/
Disallow /search/
Disallow /user/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /?q=admin%2F
Disallow /?q=comment%2Freply%2F
Disallow /?q=filter%2Ftips%2F
Disallow /?q=node%2Fadd%2F
Disallow /?q=search%2F
Disallow /?q=user%2Fpassword%2F
Disallow /?q=user%2Fregister%2F
Disallow /?q=user%2Flogin%2F
Disallow /?q=user%2Flogout%2F
Disallow /search/
Disallow /sites/all/modules/*.swf
Disallow /canal-noticias.xml
Disallow /cron.php
Disallow /user/
Disallow /book/export/
Disallow /forward/
Disallow /comment/
Disallow /feed/
Disallow /comment/reply/
Disallow /popular/
Disallow /taxonomy/
Disallow /database/
Disallow /updates/
Disallow /aggregator/
Disallow /contact/
Disallow /logout/
Disallow /contact
Disallow /logout
Disallow /print/
Disallow /forward/
Disallow /sites/all/libraries/ckeditor/

Other Records

Field Value
crawl-delay 30

linguee bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

woorank

Rule Path
Disallow /

woobot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

picsearch

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

crawler.feedback

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baidu

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://objetivocastillalamancha.es/googlenews.xml
sitemap https://objetivocastillalamancha.es/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/robotstxt.html
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Extra robots.txt content, editable from
  • administration pages (admin/config/search/custom_robots)
  • Acceso a sitemap
  • no access for table sorting paths or any paths that have parameters
  • Disallow: /*?field*
  • Disallow: /*?sort*
  • Disallow: /*&sort*
  • Disallow: /*?solrsort*
  • Disallow: /*&solrsort*
  • Disallow: /*?page=
  • Disallow: /content/*/*/*?page=
  • Disallow: /contenidos/*/*/*?page=
  • Disallow: /contenidos/*/*?page=
  • Disallow: /secciones/*?page=
  • no access to profiles that are often targeted by spammers.
  • Disallow: /profile/interest/*
  • Disallow: /profile/industries/*
  • Disallow: /profile/companies/*
  • Disallow bogus aggregator pages
  • Disallow: /aggregator/*
  • Disallow project search
  • Disallow: /project/issues/search/*
  • Disallow: /project/issues/*
  • Disallow: /*/track$
  • Disallow: /*/track?page=
  • Disallow book export
  • Disallow: /book/export/*
  • Disallow pift tests
  • Disallow: /pift/retest/*
  • Disallow project subscription
  • Disallow: /project/issues/subscribe-mail/*
  • Disallow archivos
  • Disallow: /rss.xml
  • Disallow: /sites/default/files/NOTICIAS/*/*/*.pdf
  • Directories
  • Disallow: /tracker/
  • Disallow: /xtracker/
  • Disallow : /node/*
  • Paths (clean URLs)
  • No permitir Bots
  • User-agent: SentiBot
  • Disallow: /
  • User-agent: PetalBot
  • Disallow: /
  • User-agent: trendictionbot
  • Disallow: /
  • User-agent: trendiction
  • Disallow: /
  • User-agent: serpstatbot
  • Disallow: /
Back to top