alimarket.es
robots.txt
Robots Exclusion Standard data for alimarket.es
Resource Scan
Scan Details
| Site Domain | alimarket.es |
| Base Domain | alimarket.es |
| Scan Status | Ok |
| Last Scan | 2024-09-15T07:37:35+00:00 |
| Next Scan | 2024-10-15T07:37:35+00:00 |
Last Scan
| Scanned | 2024-09-15T07:37:35+00:00 |
| URL | https://alimarket.es/robots.txt |
| Redirect | https://www.alimarket.es/robots.txt |
| Redirect Domain | www.alimarket.es |
| Redirect Base | alimarket.es |
| Domain IPs | 178.33.112.157 |
| Redirect IPs | 178.33.112.157 |
| Response IP | 178.33.112.157 |
| Found | Yes |
| Hash | 344cf18f77380229bf225eafd0d5b91e09a2a2ac25574067025399e949ba118f |
| SimHash | f37489c64d38 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /alertas/ |
| Disallow | /alerts/ |
| Disallow | /areas/ |
| Disallow | /banners/ |
| Disallow | /banners.bk/ |
| Disallow | /buscadores/ |
| Disallow | /cache/ |
| Disallow | /comun/ |
| Disallow | /empresas/ |
| Disallow | /faqs/ |
| Disallow | /hemeroteca/ |
| Disallow | /index/ |
| Disallow | /informes/ |
| Disallow | /nosotros/ |
| Disallow | /noticias/ |
| Disallow | /portadas/ |
| Disallow | /props/ |
| Disallow | /site/ |
| Disallow | /static/ |
| Disallow | /suscripciones/ |
| Disallow | /*.pdf$ |
| Disallow | /*.doc$ |
| Disallow | /*.docx$ |
| Disallow | /content_pdf*$ |
| Disallow | /*noticia_pdf/ |
| Disallow | /*informe_pdf/ |
| Disallow | /*empresa_pdf/ |
| Disallow | /*establecimiento_pdf/ |
| Disallow | /*marca_pdf/ |
| Disallow | /*obra_pdf/ |
| Disallow | /*agenda_pdf/ |
| Disallow | /*insignia_pdf/ |
*
| Rule | Path |
|---|---|
| Disallow | /blog/feed/ |
black hole
cherrypicker
disco pump 3.1
emailcollector
emailsiphon
emailwolf
extractorpro
| Rule | Path |
|---|---|
| Disallow | / |
teleport
backdoorbot/1.0
offline explorer
superbot
superbot/2.6
teleport
titan
webbandit
webcopier
webcopy
webmirror
webreaper
website extractor
winhttrack
news bot
news bot /2.1
ezooms
ezooms/1.0
mail.ru_bot/2.0
j12bot/v1.4.3
acoonbot
acoonbot/4.11.1
exabot
exabot/3.0
obot
obot/2.3.1
acoonbot
ahrefsbot
diffbot
diffbot/0.1
spinn3r
spinn3r 3.1
trendkite-akashic-crawler
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.alimarket.es/sitemap_index.xml |
Warnings
- 2 invalid lines.
Comments