Log In Sign Up

elpais.do
robots.txt

Robots Exclusion Standard data for elpais.do

Resource Scan

Scan Details

Site Domain elpais.do
Base Domain elpais.do
Scan Status Ok
Last Scan2024-11-14T13:30:51+00:00
Next Scan 2024-11-21T13:30:51+00:00

Last Scan

Scanned2024-11-14T13:30:51+00:00
URL https://elpais.do/robots.txt
Domain IPs 162.213.251.217
Response IP 162.213.251.217
Found Yes
Hash 9cae71f156daedceae0504acac722384d60ce2c6305fc703c05c4800e0cddc23
SimHash c7d15bc2c6b7

Groups

scrapy

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-movile

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

yandex

Rule Path
Allow /

bingbot

Rule Path
Allow /

ubicrawler

Rule Path
Allow /

doc

Rule Path
Allow /

sitecheck.internetseer.com

Rule Path
Allow /

zealbot

Rule Path
Allow /

msiecrawler

Rule Path
Allow /

sitesnagger

Rule Path
Allow /

webstripper

Rule Path
Allow /

webcopier

Rule Path
Allow /

fetch

Rule Path
Allow /

offline explorer

Rule Path
Allow /

teleport

Rule Path
Allow /

teleportpro

Rule Path
Allow /

webzip

Rule Path
Allow /

linko

Rule Path
Allow /

httrack

Rule Path
Allow /

microsoft.url.control

Rule Path
Allow /

xenu

Rule Path
Allow /

larbin

Rule Path
Allow /

libwww

Rule Path
Allow /

zyborg

Rule Path
Allow /

download ninja

Rule Path
Allow /

slurp

Rule Path
Allow /

maxthon

Rule Path
Allow /

cncdialer

Rule Path
Allow /

flipboardproxy

Rule Path
Allow /

flipboard

Rule Path
Allow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /wp-signup.php
Disallow /remote-login.php
Back to top

Other Records

Field Value
sitemap https://elpais.do/sitemap_index.xml
Back to top

Comments

  • Sitemap archive
Back to top