Log In Sign Up

mediotiempo.com
robots.txt

Robots Exclusion Standard data for mediotiempo.com

Resource Scan

Scan Details

Site Domain mediotiempo.com
Base Domain mediotiempo.com
Scan Status Ok
Last Scan2024-06-27T19:37:20+00:00
Next Scan 2024-07-04T19:37:20+00:00

Last Scan

Scanned2024-06-27T19:37:20+00:00
URL https://mediotiempo.com/robots.txt
Redirect https://www.mediotiempo.com/robots.txt
Redirect Domain www.mediotiempo.com
Redirect Base mediotiempo.com
Domain IPs 108.156.224.10, 108.156.224.112, 108.156.224.116, 108.156.224.93
Redirect IPs 13.33.30.129, 13.33.30.47, 13.33.30.60, 13.33.30.65
Response IP 13.33.30.129
Found Yes
Hash 53eddb22bb5d98356a21fc7c8d3915fcdb53a00bb7b2f48e498086b03cce3d66
SimHash 3016975335a6

Groups

*

Rule Path
Disallow /none

Other Records

Field Value
crawl-delay 10

genio

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

scooperbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

facebot

Rule Path
Disallow /

luminatebot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

yeti

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

gozaikbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

apache-httpclient

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

buck

Rule Path
Disallow /

wikido

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

zend_http_client

Rule Path
Disallow /

robots

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

g-i-g-a-b-o-t

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

caam

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

jersey

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

omgili

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.mediotiempo.com/sitemap/sitemap-google-news-index.xml
sitemap https://www.mediotiempo.com/sitemap/sitemap-articles-index.xml
sitemap https://www.mediotiempo.com/sitemap/sitemap-images-index.xml
sitemap https://www.mediotiempo.com/sitemap/sitemap-videos-index.xml
sitemap https://www.mediotiempo.com/sitemap/sitemap-tags-index.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
Back to top