noticiasusodidactico.com
robots.txt

Robots Exclusion Standard data for noticiasusodidactico.com

Resource Scan

Scan Details

Site Domain noticiasusodidactico.com
Base Domain noticiasusodidactico.com
Scan Status Ok
Last Scan2025-06-02T11:21:53+00:00
Next Scan 2025-06-09T11:21:53+00:00

Last Scan

Scanned2025-06-02T11:21:53+00:00
URL https://noticiasusodidactico.com/robots.txt
Domain IPs 35.214.245.218
Response IP 35.214.245.218
Found Yes
Hash ce0a66a6f5cc64a81ef5a93a80f35e6e9c4efd6de3988f6bc9bdeb1d0211bcb1
SimHash 5b4300940675

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /wp-admin
Disallow /wp-includes
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow */trackback/
Disallow */feed/
Disallow /*/feed/rss/$
Disallow /category/*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

httrack

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

mediapartners-google*

Rule Path
Allow /*

Comments

  • allow adsense bot on entire site