businessinsider.es
robots.txt
Robots Exclusion Standard data for businessinsider.es
Resource Scan
Scan Details
| Site Domain | businessinsider.es |
| Base Domain | businessinsider.es |
| Scan Status | Ok |
| Last Scan | 2024-05-26T06:42:15+00:00 |
| Next Scan | 2024-06-02T06:42:15+00:00 |
Last Scan
| Scanned | 2024-05-26T06:42:15+00:00 |
| URL | https://businessinsider.es/robots.txt |
| Redirect | https://www.businessinsider.es/robots.txt |
| Redirect Domain | www.businessinsider.es |
| Redirect Base | businessinsider.es |
| Domain IPs | 154.47.23.177, 212.102.42.89, 2a02:6ea0:d342::4, 2a02:6ea0:d638::4 |
| Redirect IPs | 154.47.23.177, 212.102.42.89, 2a02:6ea0:d342::4, 2a02:6ea0:d638::4 |
| Response IP | 212.102.42.89 |
| Found | Yes |
| Hash | 72d9abdd3a78847f60dc77208906e999af56f91aa33eee63b46058e4fd74beb4 |
| SimHash | 281115184f74 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /themes/businessinsider/css/style.css |
| Allow | /themes/businessinsider/js/ |
| Disallow | /includes/ |
| Disallow | /misc/ |
| Disallow | /modules/ |
| Disallow | /profiles/ |
| Disallow | /scripts/ |
| Disallow | /sites/all/modules/ |
| Disallow | /sites/all/themes/ |
| Disallow | /user/login |
| Disallow | /user/password |
| Disallow | /user/register |
| Disallow | /buscar |
| Disallow | /search |
| Disallow | /bm-descriptor/ |
| Disallow | /content-types/ |
| Disallow | /CHANGELOG.txt |
| Disallow | /cron.php |
| Disallow | /INSTALL.mysql.txt |
| Disallow | /INSTALL.pgsql.txt |
| Disallow | /install.php |
| Disallow | /INSTALL.txt |
| Disallow | /LICENSE.txt |
| Disallow | /MAINTAINERS.txt |
| Disallow | /update.php |
| Disallow | /UPGRADE.txt |
| Disallow | /xmlrpc.php |
| Disallow | /admin/ |
| Disallow | /comment/reply/ |
| Disallow | /logout/ |
| Disallow | /node/add/ |
| Disallow | /user/* |
| Disallow | /comment/ |
| Disallow | /image_captcha/ |
| Disallow | /social-links/ |
| Disallow | /en/ |
| Disallow | /noticia/ |
| Disallow | /lista/ |
| Disallow | /tags/titan-desert |
| Disallow | /tags/mountain-bike |
| Disallow | /tags/desafios-deportivos |
| Disallow | /tags/deporte-extremo |
| Disallow | /module/ |
| Disallow | /mercados/ |
| Disallow | /products/ |
| Disallow | /hackeo-jason-momoa-afecta-empresa-espanola-486515 |
| Disallow | /?q=admin%2F |
| Disallow | /?q=comment%2Freply%2F |
| Disallow | /?q=logout%2F |
| Disallow | /?q=node%2Fadd%2F |
| Disallow | /?q=search%2F |
| Disallow | /?q=user%2Fpassword%2F |
| Disallow | /?q=user%2Fregister%2F |
| Disallow | /?q=user%2Flogin%2F |
| Disallow | /*/www.pocketmath.com |
| Disallow | /*/revealmobile.com |
| Disallow | /*/www.liveramp.com |
| Disallow | /*/www.indexexchange.com |
| Disallow | /*/www.tresensa.com |
| Disallow | /*/www.adsquare.com |
| Disallow | /*/www.pocketmath.com |
| Disallow | /*/delicast.com |
| Disallow | /*/www.trefectamobility.com |
| Disallow | /*/vuble.tv |
| Disallow | /*/freewheel.tv |
| Disallow | /*/privacy.audienceproject.com |
| Disallow | /*/adrollgroup.com |
| Disallow | /*/contenido_final |
| Disallow | /*/seccion |
| Disallow | /*/www.parsec.media |
| Disallow | /*/components |
| Disallow | /*/home |
| Disallow | /*/otros/ |
| Disallow | /10-declaraciones-polemicas-politicos-empresarios-multimillonarios-1056861 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.businessinsider.es/sitemap.xml |
| sitemap | https://www.businessinsider.es/sitemap-video.xml |
| sitemap | https://www.businessinsider.es/sitemap-image.xml |
| sitemap | https://www.businessinsider.es/sitemapnews.xml |
| sitemap | https://www.businessinsider.es/sitemap.xml |
| sitemap | https://www.businessinsider.es/sitemap.xml |
Comments