Log In Sign Up

cdn2.excelsior.com.mx
robots.txt

Robots Exclusion Standard data for cdn2.excelsior.com.mx

Resource Scan

Scan Details

Site Domain cdn2.excelsior.com.mx
Base Domain excelsior.com.mx
Scan Status Ok
Last Scan2024-06-08T03:39:07+00:00
Next Scan 2024-06-15T03:39:07+00:00

Last Scan

Scanned2024-06-08T03:39:07+00:00
URL https://cdn2.excelsior.com.mx/robots.txt
Redirect https://www.excelsior.com.mx/robots.txt
Redirect Domain www.excelsior.com.mx
Redirect Base excelsior.com.mx
Domain IPs 18.172.170.117, 18.172.170.122, 18.172.170.95, 18.172.170.98, 2600:9000:234c:2c00:1c:ecc6:7c80:93a1, 2600:9000:234c:7000:1c:ecc6:7c80:93a1, 2600:9000:234c:7200:1c:ecc6:7c80:93a1, 2600:9000:234c:7c00:1c:ecc6:7c80:93a1, 2600:9000:234c:a600:1c:ecc6:7c80:93a1, 2600:9000:234c:c200:1c:ecc6:7c80:93a1, 2600:9000:234c:d400:1c:ecc6:7c80:93a1, 2600:9000:234c:da00:1c:ecc6:7c80:93a1
Redirect IPs 201.175.0.195
Response IP 201.175.0.195
Found Yes
Hash 5f61136d5a2b814af1ecdcab166637e9932141149133ebaf5366e4e465fe5b4e
SimHash b8947b40ee78

Groups

twitterbot

Rule Path
Disallow *
Allow /media/

*

Rule Path
Allow /.well-known/amphtml/apikey.pub

*

Rule Path
Allow /misc/*.css$
Allow /misc/*.css?
Allow /misc/*.js$
Allow /misc/*.js?
Allow /misc/*.gif
Allow /misc/*.jpg
Allow /misc/*.jpeg
Allow /misc/*.png
Allow /modules/*.css$
Allow /modules/*.css?
Allow /modules/*.js$
Allow /modules/*.js?
Allow /modules/*.gif
Allow /modules/*.jpg
Allow /modules/*.jpeg
Allow /modules/*.png
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add/
Disallow /search/
Disallow /user/register/
Disallow /user/password/
Disallow /user/login/
Disallow /user/logout/
Disallow /?q=admin%2F
Disallow /?q=comment%2Freply%2F
Disallow /?q=filter%2Ftips%2F
Disallow /?q=node%2Fadd%2F
Disallow /?q=search%2F
Disallow /?q=user%2Fpassword%2F
Disallow /?q=user%2Fregister%2F
Disallow /?q=user%2Flogin%2F
Disallow /?q=user%2Flogout%2F
Disallow /feeds/
Disallow /api/
Disallow /feeds/*
Disallow /api/*
Disallow /esi/
Disallow /esi/*
Disallow /esi/rubrikk/api/api.php
Disallow /file
Disallow /file/*
Allow /clasificados/*.js
Allow /clasificados/*.css
Allow /clasificados/*.jpg
Allow /clasificados/*?browse=
Disallow /clasificados/*--*.html
Disallow /clasificados/*%3D
Disallow /clasificados/Click
Disallow /clasificados/*--expensive
Disallow /clasificados/*--fair-price
Disallow /clasificados/*--good-price
Disallow /clasificados/*--no-price-rating
Disallow /clasificados/*--super-price
Disallow /clasificados/*expensive--
Disallow /clasificados/*fair-price--
Disallow /clasificados/*good-price--
Disallow /clasificados/*no-price-rating--
Disallow /clasificados/*super-price--
Disallow /clasificados/*--a-bit-pricy
Disallow /clasificados/*a-bit-pricy--
Disallow /clasificados/*itemindex*
Disallow /clasificados/SaveAd/
Disallow /clasificados/confirm-search/
Disallow /clasificados/get-filter-results/
Disallow /clasificados/get-top-box-results/
Disallow /clasificados/login-saved-searches/
Disallow /clasificados/manage-saved-searches/
Disallow /clasificados/recover-password/
Disallow /clasificados/register-saved-searches/
Disallow /clasificados/SavedSearch/
Disallow /clasificados/ss-logout/
Disallow /clasificados/Subscriptions/
Disallow /clasificados/update-frequency/
Disallow /clasificados/update-password/
Disallow /clasificados/Account/
Disallow /clasificados/SingleAdPage/
Disallow /clasificados/SingleAdPage/Home/
Disallow /clasificados/SingleAdPage/Product/
Disallow /clasificados/SingleAdPage/Similar/
Disallow /clasificados/*ajax.svc*
Disallow /clasificados/*_nif
Disallow /clasificados/admanagement/*
Disallow /clasificados/Controls/*
Disallow /clasificados/data/*
Disallow /clasificados/external-login/
Disallow /clasificados/ExternalApi/
Disallow /clasificados/get-results/*
Disallow /clasificados/JsonProvider/Components
Disallow /clasificados/mobile/search/
Disallow /clasificados/Report/
Disallow /clasificados/Rss/
Disallow /clasificados/set-admin-cookie/
Disallow /clasificados/SitemapHandler/
Disallow /clasificados/PartnerIntegration/
Disallow /clasificados/PopularSearches/
Disallow /clasificados/XmlProvider/
Disallow /clasificados/api/
Disallow /clasificados/*.aspx
Disallow /clasificados/browse/*
Disallow /clasificados/Reviews/
Disallow /clasificados/*?
Disallow /clasificados/serp/

ahrefsbot

Rule Path
Disallow /clasificados/

blexbot

Rule Path
Disallow /clasificados/

bloglines/3.1

Rule Path
Disallow /clasificados/

cityreview

Rule Path
Disallow /clasificados/

doc

Rule Path
Disallow /clasificados/

dotbot

Rule Path
Disallow /clasificados/

download ninja

Rule Path
Disallow /clasificados/

exabot

Rule Path
Disallow /clasificados/

fetch

Rule Path
Disallow /clasificados/

grapeshot

Rule Path
Disallow /clasificados/

grub-client

Rule Path
Disallow /clasificados/

httrack

Rule Path
Disallow /clasificados/

jyxobot/1

Rule Path
Disallow /clasificados/

k2spider

Rule Path
Disallow /clasificados/

larbin

Rule Path
Disallow /clasificados/

libwww

Rule Path
Disallow /clasificados/

linko

Rule Path
Disallow /clasificados/

microsoft.url.control

Rule Path
Disallow /clasificados/

msiecrawler

Rule Path
Disallow /clasificados/

npbot

Rule Path
Disallow /clasificados/

offline explorer

Rule Path
Disallow /clasificados/

petalbot

Rule Path
Disallow /clasificados/

proximic

Rule Path
Disallow /clasificados/

psbot

Rule Path
Disallow /clasificados/

semrushbot

Rule Path
Disallow /clasificados/

sitecheck.internetseer.com

Rule Path
Disallow /clasificados/

sitesnagger

Rule Path
Disallow /clasificados/

speedy

Rule Path
Disallow /clasificados/

teleport

Rule Path
Disallow /clasificados/

teleportpro

Rule Path
Disallow /clasificados/

ubicrawler

Rule Path
Disallow /clasificados/

webcopier

Rule Path
Disallow /clasificados/

webreaper

Rule Path
Disallow /clasificados/

webstripper

Rule Path
Disallow /clasificados/

webzip

Rule Path
Disallow /clasificados/

wget

Rule Path
Disallow /clasificados/

xenu

Rule Path
Disallow /clasificados/

zao

Rule Path
Disallow /clasificados/

zealbot

Rule Path
Disallow /clasificados/

zyborg

Rule Path
Disallow /clasificados/
Back to top

Other Records

Field Value
sitemap https://www.excelsior.com.mx/sitemap.xml
sitemap https://www.excelsior.com.mx/googlenews.xml
Back to top

Comments

  • robots.txt
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Feeds
  • file
Back to top