motormundial.es
robots.txt

Robots Exclusion Standard data for motormundial.es

Resource Scan

Scan Details

Site Domain motormundial.es
Base Domain motormundial.es
Scan Status Ok
Last Scan2024-05-26T11:02:39+00:00
Next Scan 2024-06-02T11:02:39+00:00

Last Scan

Scanned2024-05-26T11:02:39+00:00
URL https://www.motormundial.es/robots.txt
Domain IPs 104.21.77.46, 172.67.204.122, 2606:4700:3035::6815:4d2e, 2606:4700:3037::ac43:cc7a
Response IP 172.67.204.122
Found Yes
Hash abb2a2e1286d1db9bb3a9b76686c1b67f8fdf61f62dc45208184b45da3b0d7fb
SimHash 6841da808095

Groups

*

Rule Path
Allow /
Disallow /enlaces
Disallow /4423438
Allow /wp-content/uploads/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /wp-admin/
Disallow /wp-
Disallow /?s=
Disallow /search
Disallow /comparativa/
Disallow /2012-precios/
Disallow /2013-precios/
Disallow /feed
Disallow /comments/feed
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20