historiademexicobreve.com
robots.txt

Robots Exclusion Standard data for historiademexicobreve.com

Resource Scan

Scan Details

Site Domain historiademexicobreve.com
Base Domain historiademexicobreve.com
Scan Status Ok
Last Scan2024-11-14T03:00:40+00:00
Next Scan 2024-11-21T03:00:40+00:00

Last Scan

Scanned2024-11-14T03:00:40+00:00
URL https://historiademexicobreve.com/robots.txt
Domain IPs 148.163.113.26
Response IP 148.163.113.26
Found Yes
Hash dae653652f23840020109857154499442bb1338395fadf81517cbdc0318f9f4d
SimHash 42445991c6d3

Groups

*

Rule Path
Allow /wp-content/*.css$
Allow /wp-content/*.jpg$
Allow /wp-content/*.js$
Allow /wp-content/uploads/
Disallow /cgi-bin
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /wp-content/themes
Disallow /trackback
Disallow /*.js$
Disallow /comments
Disallow /author
Disallow /category
Disallow /feed/
Disallow /trackback/
Disallow /*.inc$
Disallow /*.css$
Disallow /*.php$
Disallow /tag/
Disallow /*/*/*/page/
Disallow /*/*/page/
Disallow /*/page/
Disallow /page/
Disallow /search/
Disallow /category/
Disallow /pages/
Disallow /rss.xml
Disallow /wp-login.php

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

ahrefsbot

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://www.historiademexicobreve.com/sitemap.xml