Log In Sign Up

excelsior.com.mx
robots.txt

Robots Exclusion Standard data for excelsior.com.mx

Resource Scan

Scan Details

Site Domain excelsior.com.mx
Base Domain excelsior.com.mx
Scan Status Ok
Last Scan2024-10-07T13:46:52+00:00
Next Scan 2024-10-14T13:46:52+00:00

Last Scan

Scanned2024-10-07T13:46:52+00:00
URL https://excelsior.com.mx/robots.txt
Redirect https://www.excelsior.com.mx/robots.txt
Redirect Domain www.excelsior.com.mx
Redirect Base excelsior.com.mx
Domain IPs 18.211.241.228
Redirect IPs 151.101.130.217, 151.101.194.217, 151.101.2.217, 151.101.66.217
Response IP 199.232.46.217
Found Yes
Hash 3ed1716414f9655cdf3d05a104c0a93c4d72b78e7a4535b5033227536a26321c
SimHash b8947b48ce79

Groups

twitterbot

Rule Path
Disallow *
Allow /media/

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Allow /.well-known/amphtml/apikey.pub

*

Rule Path
Allow /misc/*.css$
Allow /misc/*.css?
Allow /misc/*.js$
Allow /misc/*.js?
Allow /misc/*.gif
Allow /misc/*.jpg
Allow /misc/*.jpeg
Allow /misc/*.png
Allow /modules/*.css$
Allow /modules/*.css?
Allow /modules/*.js$
Allow /modules/*.js?
Allow /modules/*.gif
Allow /modules/*.jpg
Allow /modules/*.jpeg
Allow /modules/*.png
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /vota-cumbreimagen-2024
Disallow /?q=admin%2F
Disallow /?q=comment%2Freply%2F
Disallow /?q=filter%2Ftips%2F
Disallow /?q=node%2Fadd%2F
Disallow /?q=search%2F
Disallow /?q=user%2Fpassword%2F
Disallow /?q=user%2Fregister%2F
Disallow /?q=user%2Flogin%2F
Disallow /?q=user%2Flogout%2F
Disallow /feeds/
Disallow /api/
Disallow /feeds/*
Disallow /api/*
Disallow /esi/
Disallow /esi/*
Disallow /esi/rubrikk/api/api.php
Disallow /taxonomy-term-json
Disallow /taxonomy-term-json/*
Disallow /nacional-json
Disallow /nacional-json/*
Disallow /funcion-json
Disallow /funcion-json/*
Disallow /comunidad-json
Disallow /comunidad-json/*
Disallow /expresiones-json
Disallow /expresiones-json/*
Disallow /hacker-json
Disallow /hacker-json/*
Disallow /global-json
Disallow /global-json/*
Disallow /especial-json
Disallow /especial-json/*
Disallow /taxonomy-term-json-adrenalina
Disallow /taxonomy-term-json-adrenalina/*
Disallow /taxonomy-term-json-tv
Disallow /taxonomy-term-json-tv/*
Disallow /file
Disallow /file/*
Allow /clasificados/*.js
Allow /clasificados/*.css
Allow /clasificados/*.jpg
Allow /clasificados/*?browse=
Disallow /clasificados/*--*.html
Disallow /clasificados/*%3D
Disallow /clasificados/Click
Disallow /clasificados/*--expensive
Disallow /clasificados/*--fair-price
Disallow /clasificados/*--good-price
Disallow /clasificados/*--no-price-rating
Disallow /clasificados/*--super-price
Disallow /clasificados/*expensive--
Disallow /clasificados/*fair-price--
Disallow /clasificados/*good-price--
Disallow /clasificados/*no-price-rating--
Disallow /clasificados/*super-price--
Disallow /clasificados/*--a-bit-pricy
Disallow /clasificados/*a-bit-pricy--
Disallow /clasificados/*itemindex*
Disallow /clasificados/SaveAd/
Disallow /clasificados/confirm-search/
Disallow /clasificados/get-filter-results/
Disallow /clasificados/get-top-box-results/
Disallow /clasificados/login-saved-searches/
Disallow /clasificados/manage-saved-searches/
Disallow /clasificados/recover-password/
Disallow /clasificados/register-saved-searches/
Disallow /clasificados/SavedSearch/
Disallow /clasificados/ss-logout/
Disallow /clasificados/Subscriptions/
Disallow /clasificados/update-frequency/
Disallow /clasificados/update-password/
Disallow /clasificados/Account/
Disallow /clasificados/SingleAdPage/
Disallow /clasificados/SingleAdPage/Home/
Disallow /clasificados/SingleAdPage/Product/
Disallow /clasificados/SingleAdPage/Similar/
Disallow /clasificados/*ajax.svc*
Disallow /clasificados/*_nif
Disallow /clasificados/admanagement/*
Disallow /clasificados/Controls/*
Disallow /clasificados/data/*
Disallow /clasificados/external-login/
Disallow /clasificados/ExternalApi/
Disallow /clasificados/get-results/*
Disallow /clasificados/JsonProvider/Components
Disallow /clasificados/mobile/search/
Disallow /clasificados/Report/
Disallow /clasificados/Rss/
Disallow /clasificados/set-admin-cookie/
Disallow /clasificados/SitemapHandler/
Disallow /clasificados/PartnerIntegration/
Disallow /clasificados/PopularSearches/
Disallow /clasificados/XmlProvider/
Disallow /clasificados/api/
Disallow /clasificados/*.aspx
Disallow /clasificados/browse/*
Disallow /clasificados/Reviews/
Disallow /clasificados/*?
Disallow /clasificados/serp/

ahrefsbot

Rule Path
Disallow /clasificados/

blexbot

Rule Path
Disallow /clasificados/

bloglines/3.1

Rule Path
Disallow /clasificados/

cityreview

Rule Path
Disallow /clasificados/

doc

Rule Path
Disallow /clasificados/

dotbot

Rule Path
Disallow /clasificados/

download ninja

Rule Path
Disallow /clasificados/

exabot

Rule Path
Disallow /clasificados/

fetch

Rule Path
Disallow /clasificados/

grapeshot

Rule Path
Disallow /clasificados/

grub-client

Rule Path
Disallow /clasificados/

httrack

Rule Path
Disallow /clasificados/

jyxobot/1

Rule Path
Disallow /clasificados/

k2spider

Rule Path
Disallow /clasificados/

larbin

Rule Path
Disallow /clasificados/

libwww

Rule Path
Disallow /clasificados/

linko

Rule Path
Disallow /clasificados/

microsoft.url.control

Rule Path
Disallow /clasificados/

msiecrawler

Rule Path
Disallow /clasificados/

npbot

Rule Path
Disallow /clasificados/

offline explorer

Rule Path
Disallow /clasificados/

petalbot

Rule Path
Disallow /clasificados/

proximic

Rule Path
Disallow /clasificados/

psbot

Rule Path
Disallow /clasificados/

semrushbot

Rule Path
Disallow /clasificados/

sitecheck.internetseer.com

Rule Path
Disallow /clasificados/

sitesnagger

Rule Path
Disallow /clasificados/

speedy

Rule Path
Disallow /clasificados/

teleport

Rule Path
Disallow /clasificados/

teleportpro

Rule Path
Disallow /clasificados/

ubicrawler

Rule Path
Disallow /clasificados/

webcopier

Rule Path
Disallow /clasificados/

webreaper

Rule Path
Disallow /clasificados/

webstripper

Rule Path
Disallow /clasificados/

webzip

Rule Path
Disallow /clasificados/

wget

Rule Path
Disallow /clasificados/

xenu

Rule Path
Disallow /clasificados/

zao

Rule Path
Disallow /clasificados/

zealbot

Rule Path
Disallow /clasificados/

zyborg

Rule Path
Disallow /clasificados/
Back to top

Other Records

Field Value
sitemap https://www.excelsior.com.mx/sitemap.xml
sitemap https://www.excelsior.com.mx/googlenews.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Feeds
  • file
Back to top